2026-09-18T22:00:13.155203350Z [disk usage checker] Started
2026-09-18T22:00:13.158515509Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:13.366036831Z Gathering data for ns/openshift-cluster-version...
2026-09-18T22:00:13.519993208Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T22:00:13.672045226Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T22:00:13.773688978Z Gathering data for ns/default...
2026-09-18T22:00:14.158395599Z Gathering data for ns/openshift...
2026-09-18T22:00:14.653249910Z Gathering data for ns/kube-system...
2026-09-18T22:00:15.539204155Z Gathering data for ns/openshift-etcd...
2026-09-18T22:00:15.961433380Z Wrote inspect data to must-gather.
2026-09-18T22:00:15.961646360Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T22:00:15.961646360Z     namespaces "assisted-installer" not found
2026-09-18T22:00:16.052355053Z Waiting on subprocesses to finish execution.
2026-09-18T22:00:16.060209270Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-18T22:00:16.063598655Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-18T22:00:16.064517596Z INFO: Gathering HAProxy config files
2026-09-18T22:00:16.070087110Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-18T22:00:16.071496285Z Executing Istio gather script
2026-09-18T22:00:16.084760042Z INFO: Waiting for node performance related collection to complete ...
2026-09-18T22:00:16.112854424Z INFO: Collecting host service logs for crio
2026-09-18T22:00:16.113348252Z INFO: Collecting host service logs for kubelet
2026-09-18T22:00:16.113893965Z INFO: Collecting host service logs for rpm-ostreed
2026-09-18T22:00:16.114442527Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-18T22:00:16.117223591Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-18T22:00:16.117618117Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-18T22:00:16.118026138Z INFO: Collecting host service logs for NetworkManager
2026-09-18T22:00:16.119368892Z INFO: Collecting host service logs for openvswitch
2026-09-18T22:00:16.119788537Z INFO: Collecting host service logs for ovs-configuration
2026-09-18T22:00:16.120151242Z INFO: Collecting host service logs for ovsdb-server
2026-09-18T22:00:16.120528526Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-18T22:00:16.120976706Z INFO: Waiting for worker host service log collection to complete ...
2026-09-18T22:00:16.537388783Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-18T22:00:16.546919489Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-28h8h
2026-09-18T22:00:16.546969074Z insights-runtime-extractor-7jsp4
2026-09-18T22:00:16.546989943Z insights-runtime-extractor-j5czk
2026-09-18T22:00:16.547009476Z insights-runtime-extractor-z6rz5
2026-09-18T22:00:16.687116457Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-18T22:00:16.987963131Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-18T22:00:16.988001683Z INFO: on-disk MachineConfig config collection complete.
2026-09-18T22:00:16.991932912Z No resources found in openshift-etcd namespace.
2026-09-18T22:00:17.437916205Z Wrote inspect data to must-gather.
2026-09-18T22:00:17.658093388Z error: the server doesn't have a resource type "performanceprofile"
2026-09-18T22:00:17.818434229Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-18T22:00:17.855331188Z ERROR: No running kube-apiserver pods found
2026-09-18T22:00:18.043230974Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-18T22:00:18.043230974Z See 'oc rsync -h' for help and examples
2026-09-18T22:00:18.070062314Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-18T22:00:18.106358099Z Wrote inspect data to must-gather.
2026-09-18T22:00:18.143026240Z Wrote inspect data to must-gather.
2026-09-18T22:00:18.157682475Z INFO: Worker host service log collection to complete.
2026-09-18T22:00:18.163298559Z No resources found
2026-09-18T22:00:18.204386294Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:18.317879542Z Wrote inspect data to must-gather.
2026-09-18T22:00:18.458549099Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T22:00:18.511391587Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-18T22:00:18.525454491Z Wrote inspect data to must-gather.
2026-09-18T22:00:18.580436591Z Wrote inspect data to must-gather.
2026-09-18T22:00:18.598873865Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-18T22:00:18.663065640Z INFO: "metallb-operator" not detected. Skipping.
2026-09-18T22:00:18.685779537Z Wrote inspect data to must-gather.
2026-09-18T22:00:18.692291943Z No resources found
2026-09-18T22:00:18.696959294Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-18T22:00:18.705008871Z tar: Removing leading `/' from member names
2026-09-18T22:00:18.715322805Z INFO: HAProxy config collection complete.
2026-09-18T22:00:18.837279191Z INFO: 'previous-logs' folder not found on ip-10-0-129-249.ec2.internal, skipping...
2026-09-18T22:00:18.875262537Z error: the server doesn't have a resource type "machineconfigs"
2026-09-18T22:00:19.031604374Z Wrote inspect data to must-gather.
2026-09-18T22:00:19.081919565Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-18T22:00:19.181426091Z INFO: 'previous-logs' folder not found on ip-10-0-139-2.ec2.internal, skipping...
2026-09-18T22:00:19.255964393Z INFO: Getting rules from prometheus-k8s-0
2026-09-18T22:00:19.258148664Z Gathering data for ns/openshift-monitoring...
2026-09-18T22:00:19.374725479Z Wrote inspect data to must-gather.
2026-09-18T22:00:19.405178395Z Wrote inspect data to must-gather.
2026-09-18T22:00:19.510074433Z INFO: 'previous-logs' folder not found on ip-10-0-143-133.ec2.internal, skipping...
2026-09-18T22:00:19.510186546Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-18T22:00:19.510238130Z INFO: Machine Config Daemon termination log collection complete.
2026-09-18T22:00:19.626033621Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-18T22:00:19.674708280Z Wrote inspect data to must-gather.
2026-09-18T22:00:19.752434630Z INFO: Getting status/config from prometheus-k8s-0
2026-09-18T22:00:19.824996863Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-18T22:00:19.884186132Z Wrote inspect data to must-gather.
2026-09-18T22:00:20.000304080Z INFO: INTERCONNECT MODE
2026-09-18T22:00:20.000350834Z INFO: Gathering ovn-kubernetes DBs
2026-09-18T22:00:20.107868004Z Wrote inspect data to must-gather.
2026-09-18T22:00:20.169421476Z INFO: Gathering OVN_Northbound from ovnkube-node-2zpfh...
2026-09-18T22:00:20.169742669Z INFO: Gathering OVN_Northbound from ovnkube-node-6wlv2...
2026-09-18T22:00:20.169982605Z INFO: Gathering OVN_Northbound from ovnkube-node-lcc96...
2026-09-18T22:00:20.170252152Z INFO: Gathering OVN_Southbound from ovnkube-node-2zpfh...
2026-09-18T22:00:20.170518314Z INFO: Gathering OVN_Southbound from ovnkube-node-6wlv2...
2026-09-18T22:00:20.170831340Z INFO: Gathering OVN_Southbound from ovnkube-node-lcc96...
2026-09-18T22:00:20.259727804Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-18T22:00:20.446849572Z Wrote inspect data to must-gather.
2026-09-18T22:00:20.458354410Z tar: Removing leading `/' from member names
2026-09-18T22:00:20.477178105Z tar: Removing leading `/' from member names
2026-09-18T22:00:20.482590383Z tar: Removing leading `/' from member names
2026-09-18T22:00:20.523491196Z tar: Removing leading `/' from member names
2026-09-18T22:00:20.525216661Z tar: Removing leading `/' from member names
2026-09-18T22:00:20.533174813Z tar: Removing leading `/' from member names
2026-09-18T22:00:20.624147480Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-18T22:00:20.703615265Z Wrote inspect data to must-gather.
2026-09-18T22:00:20.906272676Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-18T22:00:20.952967361Z Wrote inspect data to must-gather.
2026-09-18T22:00:20.981654589Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-18T22:00:21.082612723Z INFO: Waiting for network log collection to complete ...
2026-09-18T22:00:21.082655208Z INFO: Waiting for ovnk database copies to complete ...
2026-09-18T22:00:21.083181807Z INFO: Copying ovnk databases complete.
2026-09-18T22:00:21.085173998Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-18T22:00:21.089257190Z ovnk_database_store/
2026-09-18T22:00:21.089257190Z ovnk_database_store/ovnkube-node-2zpfh_sbdb
2026-09-18T22:00:21.106251260Z ovnk_database_store/ovnkube-node-6wlv2_nbdb
2026-09-18T22:00:21.114815114Z ovnk_database_store/ovnkube-node-6wlv2_sbdb
2026-09-18T22:00:21.136392839Z ovnk_database_store/ovnkube-node-2zpfh_nbdb
2026-09-18T22:00:21.142637075Z ovnk_database_store/ovnkube-node-lcc96_sbdb
2026-09-18T22:00:21.160688187Z ovnk_database_store/ovnkube-node-lcc96_nbdb
2026-09-18T22:00:21.170134752Z INFO: Network log collection complete.
2026-09-18T22:00:21.210459468Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:21.214270181Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-18T22:00:21.215083917Z Wrote inspect data to must-gather.
2026-09-18T22:00:21.221619881Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T22:00:21.433398025Z Wrote inspect data to must-gather.
2026-09-18T22:00:21.446124207Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:21.448743137Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-18T22:00:21.682052297Z INFO: Getting status from alertmanager-main-0
2026-09-18T22:00:21.688289608Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:21.758697893Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-18T22:00:21.762091262Z INFO: Fallback to identify the container image from release info
2026-09-18T22:00:21.896941535Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T22:00:21.920048327Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:21.922640913Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-18T22:00:22.066438902Z Gathering data for ns/openshift-network-console...
2026-09-18T22:00:22.109160953Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:22.307127946Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:22.310029944Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-18T22:00:22.528953774Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-18T22:00:22.568575307Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:22.660559865Z Gathering data for ns/openshift-console-operator...
2026-09-18T22:00:22.682888777Z daemonset.apps/perf-node-gather-daemonset created
2026-09-18T22:00:22.800961139Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:22.804336539Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-18T22:00:22.846990012Z Waiting for performance profile collector pods to become ready: 1
2026-09-18T22:00:22.975037687Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:23.208017769Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:23.210745326Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-18T22:00:23.241158851Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:23.281035510Z Gathering data for ns/openshift-console...
2026-09-18T22:00:23.401775634Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:23.602913053Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:23.605412220Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-18T22:00:23.778384306Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:23.954931522Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-18T22:00:23.996286134Z Wrote inspect data to /must-gather/istio.
2026-09-18T22:00:24.001226865Z Waiting for performance profile collector pods to become ready: 2
2026-09-18T22:00:24.566186780Z Gathering data for ns/openshift-dns-operator...
2026-09-18T22:00:24.713633910Z Done executing Istio gather script
2026-09-18T22:00:24.964326137Z Gathering data for ns/openshift-dns...
2026-09-18T22:00:25.112547109Z Waiting for performance profile collector pods to become ready: 3
2026-09-18T22:00:25.811281794Z Gathering data for ns/openshift-image-registry...
2026-09-18T22:00:26.220262603Z Waiting for performance profile collector pods to become ready: 4
2026-09-18T22:00:26.441598031Z Gathering data for ns/openshift-ingress-operator...
2026-09-18T22:00:26.957493563Z Gathering data for ns/openshift-ingress...
2026-09-18T22:00:27.358592155Z Waiting for performance profile collector pods to become ready: 5
2026-09-18T22:00:27.366120852Z Gathering data for ns/openshift-ingress-canary...
2026-09-18T22:00:27.944220458Z Gathering data for ns/openshift-insights...
2026-09-18T22:00:28.246092916Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:28.470207060Z Waiting for performance profile collector pods to become ready: 6
2026-09-18T22:00:29.589601420Z Waiting for performance profile collector pods to become ready: 7
2026-09-18T22:00:30.669703585Z Gathering data for ns/kserve...
2026-09-18T22:00:30.704798662Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-18T22:00:30.980656118Z Collecting performance related data for node ip-10-0-129-249.ec2.internal
2026-09-18T22:00:30.988938553Z Collecting performance related data for node ip-10-0-143-133.ec2.internal
2026-09-18T22:00:30.997333837Z Collecting performance related data for node ip-10-0-139-2.ec2.internal
2026-09-18T22:00:32.379081718Z Gathering data for ns/openshift-config...
2026-09-18T22:00:32.778260221Z Gathering data for ns/openshift-config-managed...
2026-09-18T22:00:33.254258950Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:33.410268638Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-18T22:00:33.801353057Z Gathering data for ns/openshift-kube-apiserver...
2026-09-18T22:00:34.180991071Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-18T22:00:34.530331880Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-18T22:00:34.878999245Z Gathering data for ns/openshift-kube-scheduler...
2026-09-18T22:00:35.226970058Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-18T22:00:35.636598212Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-18T22:00:36.061214003Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-18T22:00:36.459890112Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-18T22:00:37.173429170Z Gathering data for ns/openshift-multus...
2026-09-18T22:00:38.262353836Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:39.037748508Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-18T22:00:39.550982932Z Collecting kubelet logs for node ip-10-0-129-249.ec2.internal
2026-09-18T22:00:39.552816458Z Collecting kubelet logs for node ip-10-0-143-133.ec2.internal
2026-09-18T22:00:39.554644397Z Collecting kubelet logs for node ip-10-0-139-2.ec2.internal
2026-09-18T22:00:39.898892237Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-6qs45 namespace
2026-09-18T22:00:39.904242251Z INFO: Node performance data collection complete.
2026-09-18T22:00:40.612705500Z Gathering data for ns/openshift-host-network...
2026-09-18T22:00:41.027408958Z Gathering data for ns/openshift-network-diagnostics...
2026-09-18T22:00:41.617438087Z Gathering data for ns/openshift-network-node-identity...
2026-09-18T22:00:42.144918265Z Gathering data for ns/openshift-network-operator...
2026-09-18T22:00:42.598410985Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-18T22:00:42.946777538Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-18T22:00:43.267695509Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:43.504998403Z Gathering data for ns/openshift-apiserver-operator...
2026-09-18T22:00:43.877115712Z Gathering data for ns/openshift-apiserver...
2026-09-18T22:00:44.381007507Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-18T22:00:44.726300601Z Gathering data for ns/openshift-controller-manager...
2026-09-18T22:00:45.075251762Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-18T22:00:45.839012475Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-18T22:00:46.257566916Z Gathering data for ns/openshift-service-ca-operator...
2026-09-18T22:00:46.655251536Z Gathering data for ns/openshift-service-ca...
2026-09-18T22:00:47.106767139Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-18T22:00:48.272134363Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T22:00:48.288918769Z Wrote inspect data to must-gather.
2026-09-18T22:00:48.288959585Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T22:00:48.288959585Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-18T22:00:48.420853713Z error: the server doesn't have a resource type "clusters"
2026-09-18T22:00:48.505565068Z Caches written to disk