2026-07-08T17:18:08.321020203Z [disk usage checker] Started
2026-07-08T17:18:08.324529779Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T17:18:08.523122720Z Gathering data for ns/openshift-cluster-version...
2026-07-08T17:18:08.663790517Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-08T17:18:08.741580835Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T17:18:08.813271601Z Gathering data for ns/default...
2026-07-08T17:18:09.081906045Z Gathering data for ns/openshift...
2026-07-08T17:18:09.459079520Z Gathering data for ns/kube-system...
2026-07-08T17:18:10.121553195Z Gathering data for ns/openshift-etcd...
2026-07-08T17:18:10.398398368Z Wrote inspect data to must-gather.
2026-07-08T17:18:10.398431500Z error: inspection completed with the errors occurred while gathering data:
2026-07-08T17:18:10.398431500Z     namespaces "assisted-installer" not found
2026-07-08T17:18:10.740301713Z Waiting on subprocesses to finish execution.
2026-07-08T17:18:10.744163587Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-08T17:18:10.751388984Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-08T17:18:10.756691448Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-08T17:18:10.760042845Z INFO: Gathering HAProxy config files
2026-07-08T17:18:10.762381659Z Executing Istio gather script
2026-07-08T17:18:10.813123701Z INFO: Collecting host service logs for crio
2026-07-08T17:18:10.813444804Z INFO: Collecting host service logs for kubelet
2026-07-08T17:18:10.813776151Z INFO: Collecting host service logs for rpm-ostreed
2026-07-08T17:18:10.814104240Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-08T17:18:10.814428525Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-08T17:18:10.814789543Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-08T17:18:10.815151679Z INFO: Collecting host service logs for NetworkManager
2026-07-08T17:18:10.815498284Z INFO: Collecting host service logs for openvswitch
2026-07-08T17:18:10.815852018Z INFO: Collecting host service logs for ovs-configuration
2026-07-08T17:18:10.816210188Z INFO: Collecting host service logs for ovsdb-server
2026-07-08T17:18:10.816543561Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-08T17:18:10.817049268Z INFO: Waiting for worker host service log collection to complete ...
2026-07-08T17:18:10.859642885Z INFO: Waiting for node performance related collection to complete ...
2026-07-08T17:18:11.094323872Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-jvh7x
2026-07-08T17:18:11.094379827Z insights-runtime-extractor-bnxpw
2026-07-08T17:18:11.094399616Z insights-runtime-extractor-mtqtw
2026-07-08T17:18:11.094418745Z insights-runtime-extractor-thxvw
2026-07-08T17:18:11.172553414Z No resources found in openshift-etcd namespace.
2026-07-08T17:18:11.444537110Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-08T17:18:11.444623639Z INFO: on-disk MachineConfig config collection complete.
2026-07-08T17:18:11.547620427Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-08T17:18:11.547620427Z See 'oc rsync -h' for help and examples
2026-07-08T17:18:11.649411308Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-08T17:18:11.705364183Z No resources found
2026-07-08T17:18:11.791354905Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-08T17:18:11.903045421Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-08T17:18:11.940405153Z INFO: "metallb-operator" not detected. Skipping.
2026-07-08T17:18:12.029616578Z Wrote inspect data to must-gather.
2026-07-08T17:18:12.062414579Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-08T17:18:12.234629808Z Wrote inspect data to must-gather.
2026-07-08T17:18:12.240262477Z No resources found
2026-07-08T17:18:12.468531661Z error: the server doesn't have a resource type "performanceprofile"
2026-07-08T17:18:12.722248714Z INFO: Worker host service log collection to complete.
2026-07-08T17:18:12.734401867Z Wrote inspect data to must-gather.
2026-07-08T17:18:12.739050245Z Wrote inspect data to must-gather.
2026-07-08T17:18:12.750638948Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-08T17:18:12.795169700Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T17:18:12.824346924Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-08T17:18:12.836946557Z tar: Removing leading `/' from member namesINFO: HAProxy config collection complete.
2026-07-08T17:18:12.891461193Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-08T17:18:12.930755028Z ERROR: No running kube-apiserver pods found
2026-07-08T17:18:13.004646602Z Wrote inspect data to must-gather.
2026-07-08T17:18:13.023373237Z Gathering data for ns/openshift-monitoring...
2026-07-08T17:18:13.089909803Z Wrote inspect data to must-gather.
2026-07-08T17:18:13.138365445Z Wrote inspect data to must-gather.
2026-07-08T17:18:13.161388767Z Wrote inspect data to must-gather.
2026-07-08T17:18:13.172541699Z INFO: 'previous-logs' folder not found on ip-10-0-130-180.ec2.internal, skipping...
2026-07-08T17:18:13.286236206Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-08T17:18:13.312484054Z error: the server doesn't have a resource type "machineconfigs"
2026-07-08T17:18:13.335775084Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T17:18:13.432102950Z INFO: Getting rules from prometheus-k8s-0
2026-07-08T17:18:13.436358146Z Wrote inspect data to must-gather.
2026-07-08T17:18:13.450539499Z INFO: 'previous-logs' folder not found on ip-10-0-136-180.ec2.internal, skipping...
2026-07-08T17:18:13.478998504Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-08T17:18:13.697342798Z Wrote inspect data to must-gather.
2026-07-08T17:18:13.701250787Z INFO: 'previous-logs' folder not found on ip-10-0-141-52.ec2.internal, skipping...
2026-07-08T17:18:13.701336859Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-08T17:18:13.701383237Z INFO: Machine Config Daemon termination log collection complete.
2026-07-08T17:18:13.708867355Z Wrote inspect data to must-gather.
2026-07-08T17:18:13.834318757Z INFO: Getting status/config from prometheus-k8s-0
2026-07-08T17:18:13.858651195Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-08T17:18:13.963391306Z INFO: INTERCONNECT MODE
2026-07-08T17:18:13.963474601Z INFO: Gathering ovn-kubernetes DBs
2026-07-08T17:18:14.049489640Z Wrote inspect data to must-gather.
2026-07-08T17:18:14.107535567Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-08T17:18:14.110350734Z INFO: Gathering OVN_Northbound from ovnkube-node-6xnrg...
2026-07-08T17:18:14.110654171Z INFO: Gathering OVN_Northbound from ovnkube-node-bdnd2...
2026-07-08T17:18:14.110958918Z INFO: Gathering OVN_Northbound from ovnkube-node-h5x9p...
2026-07-08T17:18:14.111282356Z INFO: Gathering OVN_Southbound from ovnkube-node-6xnrg...
2026-07-08T17:18:14.111551298Z INFO: Gathering OVN_Southbound from ovnkube-node-bdnd2...
2026-07-08T17:18:14.111840112Z INFO: Gathering OVN_Southbound from ovnkube-node-h5x9p...
2026-07-08T17:18:14.353323830Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-08T17:18:14.370104314Z Wrote inspect data to must-gather.
2026-07-08T17:18:14.429677274Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:14.447161501Z tar: Removing leading `/' from member names
2026-07-08T17:18:14.466315430Z tar: Removing leading `/' from member names
2026-07-08T17:18:14.466680284Z tar: Removing leading `/' from member names
2026-07-08T17:18:14.466680284Z tar: Removing leading `/' from member names
2026-07-08T17:18:14.474779235Z tar: Removing leading `/' from member names
2026-07-08T17:18:14.477894984Z tar: Removing leading `/' from member names
2026-07-08T17:18:14.619003049Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-08T17:18:14.620272953Z Wrote inspect data to must-gather.
2026-07-08T17:18:14.682708293Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:14.685101137Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-08T17:18:14.725600631Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-08T17:18:14.837605693Z Wrote inspect data to must-gather.
2026-07-08T17:18:14.879694207Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-08T17:18:14.895103435Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:14.900085078Z INFO: Waiting for network log collection to complete ...
2026-07-08T17:18:14.900149288Z INFO: Waiting for ovnk database copies to complete ...
2026-07-08T17:18:14.900651692Z INFO: Copying ovnk databases complete.
2026-07-08T17:18:14.902379395Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-08T17:18:14.905033845Z ovnk_database_store/
2026-07-08T17:18:14.905079622Z ovnk_database_store/ovnkube-node-bdnd2_sbdb
2026-07-08T17:18:14.930359333Z ovnk_database_store/ovnkube-node-6xnrg_sbdb
2026-07-08T17:18:14.944301147Z ovnk_database_store/ovnkube-node-bdnd2_nbdb
2026-07-08T17:18:14.952675300Z ovnk_database_store/ovnkube-node-6xnrg_nbdb
2026-07-08T17:18:14.960428097Z ovnk_database_store/ovnkube-node-h5x9p_nbdb
2026-07-08T17:18:14.968272383Z ovnk_database_store/ovnkube-node-h5x9p_sbdb
2026-07-08T17:18:14.987253811Z INFO: Network log collection complete.
2026-07-08T17:18:15.060632405Z Wrote inspect data to must-gather.
2026-07-08T17:18:15.115942515Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:15.118586609Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-08T17:18:15.123952602Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-08T17:18:15.250649844Z W0708 17:18:15.250602    1485 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-08T17:18:15.258272268Z Wrote inspect data to must-gather.
2026-07-08T17:18:15.308684214Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:15.336954680Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T17:18:15.500121608Z INFO: Getting status from alertmanager-main-0
2026-07-08T17:18:15.516022151Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:15.519437571Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-08T17:18:15.533330012Z Gathering data for ns/openshift-network-console...
2026-07-08T17:18:15.554524262Z Wrote inspect data to must-gather.
2026-07-08T17:18:15.746575171Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:15.804825371Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-08T17:18:15.807620326Z INFO: Fallback to identify the container image from release info
2026-07-08T17:18:15.949309947Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:15.952617310Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-08T17:18:15.964098267Z Gathering data for ns/openshift-console-operator...
2026-07-08T17:18:16.101457640Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:16.276784181Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:18:16.320552188Z Gathering data for ns/openshift-console...
2026-07-08T17:18:16.527995558Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-08T17:18:16.656829334Z daemonset.apps/perf-node-gather-daemonset created
2026-07-08T17:18:16.795215115Z Waiting for performance profile collector pods to become ready: 1
2026-07-08T17:18:16.835865960Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-08T17:18:16.914939866Z Done executing Istio gather script
2026-07-08T17:18:17.311692287Z Gathering data for ns/openshift-dns-operator...
2026-07-08T17:18:17.604093593Z Gathering data for ns/openshift-dns...
2026-07-08T17:18:17.908341398Z Waiting for performance profile collector pods to become ready: 2
2026-07-08T17:18:18.163175580Z Gathering data for ns/openshift-image-registry...
2026-07-08T17:18:18.340969479Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T17:18:18.622832014Z Gathering data for ns/openshift-ingress-operator...
2026-07-08T17:18:18.908780920Z Gathering data for ns/openshift-ingress...
2026-07-08T17:18:19.020504893Z Waiting for performance profile collector pods to become ready: 3
2026-07-08T17:18:19.224043403Z Gathering data for ns/openshift-ingress-canary...
2026-07-08T17:18:19.592065972Z Gathering data for ns/openshift-insights...
2026-07-08T17:18:20.139921732Z Waiting for performance profile collector pods to become ready: 4
2026-07-08T17:18:21.253122102Z Waiting for performance profile collector pods to become ready: 5
2026-07-08T17:18:21.391152886Z Gathering data for ns/openshift-config...
2026-07-08T17:18:21.628596752Z Gathering data for ns/openshift-config-managed...
2026-07-08T17:18:21.996571557Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-08T17:18:22.233676882Z Gathering data for ns/openshift-kube-apiserver...
2026-07-08T17:18:22.360714501Z Waiting for performance profile collector pods to become ready: 6
2026-07-08T17:18:22.488031696Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-08T17:18:22.791940131Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-08T17:18:23.045050448Z Gathering data for ns/openshift-kube-scheduler...
2026-07-08T17:18:23.285013341Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-08T17:18:23.345388125Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T17:18:23.470393727Z Waiting for performance profile collector pods to become ready: 7
2026-07-08T17:18:23.562199782Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-08T17:18:23.863970856Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-08T17:18:24.175235581Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-08T17:18:24.583868871Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-08T17:18:24.650234364Z Gathering data for ns/openshift-multus...
2026-07-08T17:18:24.797821770Z Collecting performance related data for node ip-10-0-136-180.ec2.internal
2026-07-08T17:18:24.804377580Z Collecting performance related data for node ip-10-0-141-52.ec2.internal
2026-07-08T17:18:24.811379012Z Collecting performance related data for node ip-10-0-130-180.ec2.internal
2026-07-08T17:18:26.327683887Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-08T17:18:27.685967378Z Gathering data for ns/openshift-host-network...
2026-07-08T17:18:28.012185623Z Gathering data for ns/openshift-network-diagnostics...
2026-07-08T17:18:28.350311688Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T17:18:28.531038852Z Gathering data for ns/openshift-network-node-identity...
2026-07-08T17:18:28.950370402Z Gathering data for ns/openshift-network-operator...
2026-07-08T17:18:29.319655646Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-08T17:18:29.572406205Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-08T17:18:30.047322661Z Gathering data for ns/openshift-apiserver-operator...
2026-07-08T17:18:30.327619144Z Gathering data for ns/openshift-apiserver...
2026-07-08T17:18:30.701423882Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-08T17:18:30.929789674Z Gathering data for ns/openshift-controller-manager...
2026-07-08T17:18:31.174973393Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-08T17:18:31.870293225Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-08T17:18:32.149791548Z Gathering data for ns/openshift-service-ca-operator...
2026-07-08T17:18:32.508579528Z Gathering data for ns/openshift-service-ca...
2026-07-08T17:18:32.574788800Z Collecting kubelet logs for node ip-10-0-136-180.ec2.internal
2026-07-08T17:18:32.577024235Z Collecting kubelet logs for node ip-10-0-141-52.ec2.internal
2026-07-08T17:18:32.579094765Z Collecting kubelet logs for node ip-10-0-130-180.ec2.internal
2026-07-08T17:18:32.890777378Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-08T17:18:32.909221192Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-b9fzb namespace
2026-07-08T17:18:32.913964343Z INFO: Node performance data collection complete.
2026-07-08T17:18:33.355891973Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T17:18:33.759389211Z Wrote inspect data to must-gather.
2026-07-08T17:18:33.759432049Z error: inspection completed with the errors occurred while gathering data:
2026-07-08T17:18:33.759432049Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-08T17:18:33.878549916Z error: the server doesn't have a resource type "clusters"
2026-07-08T17:18:34.503720579Z Caches written to disk