2026-11-03T15:46:44.169356335Z [disk usage checker] Started
2026-11-03T15:46:44.172631533Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:46:45.065837061Z Gathering data for ns/openshift-cluster-version...
2026-11-03T15:46:45.939233536Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-11-03T15:46:46.475407626Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-03T15:46:47.006826417Z Gathering data for ns/default...
2026-11-03T15:46:48.816036156Z Gathering data for ns/openshift...
2026-11-03T15:46:49.177328448Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:46:50.731109567Z Gathering data for ns/kube-system...
2026-11-03T15:46:50.916769670Z Waiting on subprocesses to finish execution.
2026-11-03T15:46:50.924900883Z INFO: Gathering HAProxy config files
2026-11-03T15:46:50.925822091Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-11-03T15:46:50.937179162Z Executing Istio gather script
2026-11-03T15:46:50.947977143Z INFO: Gathering machine config daemon's old logs from all nodes
2026-11-03T15:46:50.969798515Z INFO: Waiting for node performance related collection to complete ...
2026-11-03T15:46:50.988838816Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-11-03T15:46:51.062901064Z INFO: Collecting host service logs for crio
2026-11-03T15:46:51.063279876Z INFO: Collecting host service logs for kubelet
2026-11-03T15:46:51.063639005Z INFO: Collecting host service logs for rpm-ostreed
2026-11-03T15:46:51.063973500Z INFO: Collecting host service logs for ostree-finalize-staged
2026-11-03T15:46:51.064307143Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-11-03T15:46:51.064633659Z INFO: Collecting host service logs for machine-config-daemon-host
2026-11-03T15:46:51.064963879Z INFO: Collecting host service logs for NetworkManager
2026-11-03T15:46:51.065313493Z INFO: Collecting host service logs for openvswitch
2026-11-03T15:46:51.065629097Z INFO: Collecting host service logs for ovs-configuration
2026-11-03T15:46:51.065954566Z INFO: Collecting host service logs for ovsdb-server
2026-11-03T15:46:51.066292330Z INFO: Collecting host service logs for ovs-vswitchd
2026-11-03T15:46:51.066663706Z INFO: Waiting for worker host service log collection to complete ...
2026-11-03T15:46:51.701460426Z INFO: "metallb-operator" not detected. Skipping.
2026-11-03T15:46:51.777670682Z No resources found in openshift-etcd namespace.
2026-11-03T15:46:51.779960422Z INFO: "sriov-network-operator" not detected. Skipping.
2026-11-03T15:46:51.801106428Z INFO: Found 1 replicas - prometheus-k8s-0
2026-11-03T15:46:51.897157096Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-11-03T15:46:52.032479026Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-11-03T15:46:52.032562875Z INFO: on-disk MachineConfig config collection complete.
2026-11-03T15:46:52.048203308Z No resources found
2026-11-03T15:46:52.120662728Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-8bmfp
2026-11-03T15:46:52.120662728Z insights-runtime-extractor-6gm5s
2026-11-03T15:46:52.120662728Z insights-runtime-extractor-cn8gn
2026-11-03T15:46:52.120662728Z insights-runtime-extractor-knn95
2026-11-03T15:46:52.347034035Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-11-03T15:46:52.515366074Z Wrote inspect data to must-gather.
2026-11-03T15:46:52.743765480Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-11-03T15:46:52.743765480Z See 'oc rsync -h' for help and examples
2026-11-03T15:46:52.794306201Z error: the server doesn't have a resource type "performanceprofile"
2026-11-03T15:46:52.844999349Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-11-03T15:46:52.969988013Z INFO: Worker host service log collection to complete.
2026-11-03T15:46:53.132672715Z INFO: "nfd" not detected. Skipping.
2026-11-03T15:46:53.165885582Z Wrote inspect data to must-gather.
2026-11-03T15:46:53.191331943Z INFO: 'previous-logs' folder not found on ip-10-0-129-198.us-west-1.compute.internal, skipping...
2026-11-03T15:46:53.219137855Z Wrote inspect data to must-gather.
2026-11-03T15:46:53.223421313Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-03T15:46:53.251152087Z Wrote inspect data to must-gather.
2026-11-03T15:46:53.302042721Z ERROR: No running kube-apiserver pods found
2026-11-03T15:46:53.322802195Z No resources found
2026-11-03T15:46:53.331731595Z Wrote inspect data to must-gather.
2026-11-03T15:46:53.383577862Z INFO: Waiting for HAProxy config collection to complete ...
2026-11-03T15:46:53.404231207Z Wrote inspect data to must-gather.
2026-11-03T15:46:53.426023612Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-11-03T15:46:53.495557096Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-11-03T15:46:53.588590768Z INFO: 'previous-logs' folder not found on ip-10-0-130-92.us-west-1.compute.internal, skipping...
2026-11-03T15:46:53.661473954Z Wrote inspect data to must-gather.
2026-11-03T15:46:53.950959887Z INFO: 'previous-logs' folder not found on ip-10-0-139-187.us-west-1.compute.internal, skipping...
2026-11-03T15:46:53.951003616Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-11-03T15:46:53.951003616Z INFO: Machine Config Daemon termination log collection complete.
2026-11-03T15:46:53.984574572Z error: the server doesn't have a resource type "machineconfigs"
2026-11-03T15:46:54.082233628Z Wrote inspect data to must-gather.
2026-11-03T15:46:54.181935138Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:46:54.271300418Z tar: Removing leading `/' from member names
2026-11-03T15:46:54.325639277Z INFO: HAProxy config collection complete.
2026-11-03T15:46:54.407653769Z error: the server doesn't have a resource type "machineconfigpools"
2026-11-03T15:46:54.759942907Z INFO: Getting rules from prometheus-k8s-0
2026-11-03T15:46:54.765578122Z Wrote inspect data to must-gather.
2026-11-03T15:46:54.846534165Z Wrote inspect data to must-gather.
2026-11-03T15:46:55.061210998Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io
2026-11-03T15:46:55.273229451Z error: the server doesn't have a resource type "kubeletconfigs"
2026-11-03T15:46:55.473217510Z Wrote inspect data to must-gather.
2026-11-03T15:46:55.563794120Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:55.679858626Z Wrote inspect data to must-gather.
2026-11-03T15:46:56.085023649Z Wrote inspect data to must-gather.
2026-11-03T15:46:56.154165488Z INFO: LEGACY MODE
2026-11-03T15:46:56.154222011Z INFO: Gathering ovn-kubernetes master data
2026-11-03T15:46:56.181534319Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:56.184555776Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-11-03T15:46:56.214206371Z INFO: Getting status/config from prometheus-k8s-0
2026-11-03T15:46:56.273837088Z Gathering data for ns/openshift-cloud-credential-operator...
2026-11-03T15:46:56.520311745Z Wrote inspect data to must-gather.
2026-11-03T15:46:56.653256166Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:56.936849162Z Wrote inspect data to must-gather.
2026-11-03T15:46:57.039594965Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-11-03T15:46:57.322591507Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:57.326222094Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-11-03T15:46:57.470348956Z Wrote inspect data to must-gather.
2026-11-03T15:46:57.610497973Z INFO: Getting status/flags from prometheus-k8s-0
2026-11-03T15:46:57.626276217Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-11-03T15:46:57.870346646Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:57.883326153Z Wrote inspect data to must-gather.
2026-11-03T15:46:58.045055514Z INFO: Waiting for network log collection to complete ...
2026-11-03T15:46:58.045100768Z INFO: Waiting for ovnk database copies to complete ...
2026-11-03T15:46:58.045505425Z INFO: Copying ovnk databases complete.
2026-11-03T15:46:58.047229246Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory
2026-11-03T15:46:58.047683400Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 672 is not a child of this shell
2026-11-03T15:46:58.047683400Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 673 is not a child of this shell
2026-11-03T15:46:58.047683400Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 674 is not a child of this shell
2026-11-03T15:46:58.047683400Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 675 is not a child of this shell
2026-11-03T15:46:58.047683400Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 676 is not a child of this shell
2026-11-03T15:46:58.047683400Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 677 is not a child of this shell
2026-11-03T15:46:58.047683400Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 678 is not a child of this shell
2026-11-03T15:46:58.047718981Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 679 is not a child of this shell
2026-11-03T15:46:58.047718981Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 715 is not a child of this shell
2026-11-03T15:46:58.047718981Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 716 is not a child of this shell
2026-11-03T15:46:58.047718981Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 717 is not a child of this shell
2026-11-03T15:46:58.047718981Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 718 is not a child of this shell
2026-11-03T15:46:58.047718981Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 719 is not a child of this shell
2026-11-03T15:46:58.051346667Z tar: ovnk_database_store: Cannot stat: No such file or directory
2026-11-03T15:46:58.054947674Z tar: Exiting with failure status due to previous errors
2026-11-03T15:46:58.055442486Z INFO: Network log collection complete.
2026-11-03T15:46:58.429440672Z Wrote inspect data to must-gather.
2026-11-03T15:46:58.469367287Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-11-03T15:46:58.529771404Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:58.535344157Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-11-03T15:46:58.702197104Z Gathering data for ns/openshift-etcd...
2026-11-03T15:46:59.059186438Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:59.186737365Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:46:59.204637707Z Gathering data for ns/openshift-monitoring...
2026-11-03T15:46:59.262253314Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod
2026-11-03T15:46:59.367897103Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-11-03T15:46:59.625909652Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c
2026-11-03T15:46:59.682619011Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:46:59.685642929Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-11-03T15:47:00.001788613Z daemonset.apps/perf-node-gather-daemonset created
2026-11-03T15:47:00.209914973Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:47:00.262012166Z Waiting for performance profile collector pods to become ready: 1
2026-11-03T15:47:00.486322583Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-11-03T15:47:00.537346344Z Wrote inspect data to must-gather.
2026-11-03T15:47:00.537384018Z error: inspection completed with the errors occurred while gathering data:
2026-11-03T15:47:00.537384018Z     namespaces "assisted-installer" not found
2026-11-03T15:47:00.873472922Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:47:00.876211641Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-11-03T15:47:01.263373829Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:47:01.525630135Z Waiting for performance profile collector pods to become ready: 2
2026-11-03T15:47:01.701514672Z INFO: Getting status from alertmanager-main-0
2026-11-03T15:47:01.883989398Z Wrote inspect data to /must-gather/istio.
2026-11-03T15:47:02.778195644Z Waiting for performance profile collector pods to become ready: 3
2026-11-03T15:47:03.612327167Z Done executing Istio gather script
2026-11-03T15:47:04.027970706Z Waiting for performance profile collector pods to become ready: 4
2026-11-03T15:47:04.191660820Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:05.275109566Z Waiting for performance profile collector pods to become ready: 5
2026-11-03T15:47:06.522435672Z Waiting for performance profile collector pods to become ready: 6
2026-11-03T15:47:07.768306179Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-11-03T15:47:08.329312984Z Collecting performance related data for node ip-10-0-139-187.us-west-1.compute.internal
2026-11-03T15:47:08.336364185Z Collecting performance related data for node ip-10-0-130-92.us-west-1.compute.internal
2026-11-03T15:47:08.343582739Z Collecting performance related data for node ip-10-0-129-198.us-west-1.compute.internal
2026-11-03T15:47:09.196548276Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:14.201186689Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:19.205922044Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:20.333354330Z Gathering data for ns/openshift-network-console...
2026-11-03T15:47:21.651271621Z Collecting kubelet logs for node ip-10-0-139-187.us-west-1.compute.internal
2026-11-03T15:47:21.653059677Z Collecting kubelet logs for node ip-10-0-130-92.us-west-1.compute.internal
2026-11-03T15:47:21.654880800Z Collecting kubelet logs for node ip-10-0-129-198.us-west-1.compute.internal
2026-11-03T15:47:22.518191154Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4vbqk namespace
2026-11-03T15:47:22.572867319Z INFO: Node performance data collection complete.
2026-11-03T15:47:22.984320607Z Gathering data for ns/openshift-console-operator...
2026-11-03T15:47:24.210633767Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:25.327592772Z Gathering data for ns/openshift-console...
2026-11-03T15:47:28.445294300Z Gathering data for ns/openshift-cluster-storage-operator...
2026-11-03T15:47:29.215358318Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:31.554067977Z Gathering data for ns/openshift-dns-operator...
2026-11-03T15:47:33.620771571Z Gathering data for ns/openshift-dns...
2026-11-03T15:47:34.219945329Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:39.224768522Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:39.705162746Z Gathering data for ns/openshift-image-registry...
2026-11-03T15:47:43.307361348Z Gathering data for ns/openshift-ingress-operator...
2026-11-03T15:47:44.229503272Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:45.290945346Z Gathering data for ns/openshift-ingress...
2026-11-03T15:47:47.725141205Z Gathering data for ns/openshift-ingress-canary...
2026-11-03T15:47:49.234157459Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:50.999217818Z Gathering data for ns/openshift-insights...
2026-11-03T15:47:54.238797867Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:59.248457236Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:47:59.928708396Z Gathering data for ns/openshift-jobset-operator...
2026-11-03T15:48:02.992633487Z Gathering data for ns/openshift-config...
2026-11-03T15:48:04.253236835Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:04.827119352Z Gathering data for ns/openshift-config-managed...
2026-11-03T15:48:06.824468134Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-11-03T15:48:08.656303269Z Gathering data for ns/openshift-kube-apiserver...
2026-11-03T15:48:09.257919243Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:10.480601024Z Gathering data for ns/openshift-kube-controller-manager...
2026-11-03T15:48:12.325837153Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-11-03T15:48:14.262575535Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:14.267711213Z Gathering data for ns/openshift-kube-scheduler...
2026-11-03T15:48:16.099707337Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-11-03T15:48:18.046405451Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-11-03T15:48:19.267266131Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:20.764428073Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-11-03T15:48:23.047897466Z Gathering data for ns/openshift-user-workload-monitoring...
2026-11-03T15:48:24.271982956Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:25.823373714Z Gathering data for ns/openshift-multus...
2026-11-03T15:48:29.276713140Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:34.281383922Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:39.286085928Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:42.628836425Z Gathering data for ns/openshift-ovn-kubernetes...
2026-11-03T15:48:44.290827644Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:49.295672683Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:54.300780630Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:48:56.344162142Z Gathering data for ns/openshift-host-network...
2026-11-03T15:48:58.467181368Z Gathering data for ns/openshift-network-diagnostics...
2026-11-03T15:48:59.305950241Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:03.074406884Z Gathering data for ns/openshift-network-node-identity...
2026-11-03T15:49:04.310689907Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:06.147544104Z Gathering data for ns/openshift-network-operator...
2026-11-03T15:49:09.315357494Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:09.318530777Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-11-03T15:49:11.142313052Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-11-03T15:49:14.320100142Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:14.724369119Z Gathering data for ns/openshift-apiserver-operator...
2026-11-03T15:49:16.581258193Z Gathering data for ns/openshift-apiserver...
2026-11-03T15:49:19.118550541Z Gathering data for ns/openshift-controller-manager-operator...
2026-11-03T15:49:19.324773024Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:20.930945895Z Gathering data for ns/openshift-controller-manager...
2026-11-03T15:49:22.814330169Z Gathering data for ns/openshift-cluster-samples-operator...
2026-11-03T15:49:24.329442157Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:25.935549411Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-11-03T15:49:27.821152187Z Gathering data for ns/openshift-service-ca-operator...
2026-11-03T15:49:29.333977026Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:30.100531752Z Gathering data for ns/openshift-service-ca...
2026-11-03T15:49:32.716536704Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-11-03T15:49:34.338677748Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-11-03T15:49:38.784346599Z Wrote inspect data to must-gather.
2026-11-03T15:49:38.784391457Z error: inspection completed with the errors occurred while gathering data:
2026-11-03T15:49:38.784391457Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-11-03T15:49:39.207883705Z error: the server doesn't have a resource type "clusters"
2026-11-03T15:49:39.251938187Z Caches written to disk