2026-09-07T12:45:27.936559687Z [disk usage checker] Started
2026-09-07T12:45:27.940504226Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-07T12:45:28.141833259Z Gathering data for ns/openshift-cluster-version...
2026-09-07T12:45:28.275018700Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T12:45:28.379551895Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:45:28.464843656Z Gathering data for ns/default...
2026-09-07T12:45:28.814518791Z Gathering data for ns/openshift...
2026-09-07T12:45:29.249553745Z Gathering data for ns/kube-system...
2026-09-07T12:45:29.949665403Z Gathering data for ns/openshift-etcd...
2026-09-07T12:45:30.283188495Z Wrote inspect data to must-gather.
2026-09-07T12:45:30.283221555Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T12:45:30.283221555Z     namespaces "assisted-installer" not found
2026-09-07T12:45:30.503583275Z Waiting on subprocesses to finish execution.
2026-09-07T12:45:30.510993047Z INFO: Gathering HAProxy config files
2026-09-07T12:45:30.510993047Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-07T12:45:30.518668443Z Executing Istio gather script
2026-09-07T12:45:30.518704911Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-07T12:45:30.521958495Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-07T12:45:30.529351253Z INFO: Collecting host service logs for crio
2026-09-07T12:45:30.529623417Z INFO: Collecting host service logs for kubelet
2026-09-07T12:45:30.529915286Z INFO: Collecting host service logs for rpm-ostreed
2026-09-07T12:45:30.530182677Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-07T12:45:30.530512033Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-07T12:45:30.530809641Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-07T12:45:30.531133469Z INFO: Collecting host service logs for NetworkManager
2026-09-07T12:45:30.531714324Z INFO: Collecting host service logs for openvswitch
2026-09-07T12:45:30.532060415Z INFO: Collecting host service logs for ovs-configuration
2026-09-07T12:45:30.532415632Z INFO: Collecting host service logs for ovsdb-server
2026-09-07T12:45:30.532785701Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-07T12:45:30.533168443Z INFO: Waiting for worker host service log collection to complete ...
2026-09-07T12:45:30.594856371Z INFO: Waiting for node performance related collection to complete ...
2026-09-07T12:45:30.882111849Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-07T12:45:30.882219751Z INFO: on-disk MachineConfig config collection complete.
2026-09-07T12:45:31.074531926Z Wrote inspect data to must-gather.
2026-09-07T12:45:31.250485173Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-07T12:45:31.341565215Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-07T12:45:31.560799910Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-sbjgx
2026-09-07T12:45:31.560799910Z insights-runtime-extractor-2wjz5
2026-09-07T12:45:31.560799910Z insights-runtime-extractor-jnz4w
2026-09-07T12:45:31.560799910Z insights-runtime-extractor-srwnn
2026-09-07T12:45:31.578867497Z No resources found
2026-09-07T12:45:31.668701807Z INFO: "metallb-operator" not detected. Skipping.
2026-09-07T12:45:31.778133787Z No resources found in openshift-etcd namespace.
2026-09-07T12:45:31.814250133Z error: the server doesn't have a resource type "performanceprofile"
2026-09-07T12:45:31.865032339Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-07T12:45:31.955326436Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-07T12:45:31.974693113Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-07T12:45:32.041691504Z INFO: Worker host service log collection to complete.
2026-09-07T12:45:32.047937192Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-07T12:45:32.047937192Z See 'oc rsync -h' for help and examples
2026-09-07T12:45:32.129961872Z Wrote inspect data to must-gather.
2026-09-07T12:45:32.159429756Z ERROR: No running kube-apiserver pods found
2026-09-07T12:45:32.455411220Z Wrote inspect data to must-gather.
2026-09-07T12:45:32.600724364Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-07T12:45:32.634580328Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-07T12:45:32.659633906Z tar: Removing leading `/' from member names
2026-09-07T12:45:32.666739297Z INFO: HAProxy config collection complete.
2026-09-07T12:45:32.763802078Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:45:32.776868802Z No resources found
2026-09-07T12:45:32.792978646Z INFO: 'previous-logs' folder not found on ip-10-0-133-225.ec2.internal, skipping...
2026-09-07T12:45:32.910483613Z error: the server doesn't have a resource type "machineconfigs"
2026-09-07T12:45:32.944995582Z Gathering data for ns/openshift-monitoring...
2026-09-07T12:45:32.997159134Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-07T12:45:33.014487014Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.030603187Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.077111265Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.174297215Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.243281677Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-07T12:45:33.284977388Z INFO: 'previous-logs' folder not found on ip-10-0-137-63.ec2.internal, skipping...
2026-09-07T12:45:33.286726287Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-07T12:45:33.387824762Z INFO: Getting rules from prometheus-k8s-0
2026-09-07T12:45:33.456993965Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.492274667Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.554309096Z INFO: 'previous-logs' folder not found on ip-10-0-139-210.ec2.internal, skipping...
2026-09-07T12:45:33.554348304Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-07T12:45:33.554348304Z INFO: Machine Config Daemon termination log collection complete.
2026-09-07T12:45:33.653831114Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-07T12:45:33.738182651Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.798355863Z INFO: Getting status/config from prometheus-k8s-0
2026-09-07T12:45:33.870510098Z Wrote inspect data to must-gather.
2026-09-07T12:45:33.999872858Z Wrote inspect data to must-gather.
2026-09-07T12:45:34.099491252Z Wrote inspect data to must-gather.
2026-09-07T12:45:34.226367408Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-07T12:45:34.281763664Z INFO: INTERCONNECT MODE
2026-09-07T12:45:34.281812861Z INFO: Gathering ovn-kubernetes DBs
2026-09-07T12:45:34.323906011Z Wrote inspect data to must-gather.
2026-09-07T12:45:34.374419054Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-07T12:45:34.432060130Z INFO: Gathering OVN_Northbound from ovnkube-node-46ffx...
2026-09-07T12:45:34.432366671Z INFO: Gathering OVN_Northbound from ovnkube-node-ck4k4...
2026-09-07T12:45:34.432691344Z INFO: Gathering OVN_Northbound from ovnkube-node-lmnx5...
2026-09-07T12:45:34.432972987Z INFO: Gathering OVN_Southbound from ovnkube-node-46ffx...
2026-09-07T12:45:34.433239352Z INFO: Gathering OVN_Southbound from ovnkube-node-ck4k4...
2026-09-07T12:45:34.433585995Z INFO: Gathering OVN_Southbound from ovnkube-node-lmnx5...
2026-09-07T12:45:34.506343481Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-07T12:45:34.614427475Z Wrote inspect data to must-gather.
2026-09-07T12:45:34.678514486Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:34.716717453Z tar: Removing leading `/' from member names
2026-09-07T12:45:34.786471296Z tar: Removing leading `/' from member names
2026-09-07T12:45:34.805896989Z tar: Removing leading `/' from member names
2026-09-07T12:45:34.829033713Z tar: Removing leading `/' from member names
2026-09-07T12:45:34.870619914Z tar: Removing leading `/' from member names
2026-09-07T12:45:34.880022623Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T12:45:34.893201038Z tar: Removing leading `/' from member names
2026-09-07T12:45:34.949545163Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-07T12:45:34.952950050Z Wrote inspect data to must-gather.
2026-09-07T12:45:34.992398856Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:34.995664253Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-07T12:45:35.219526709Z Wrote inspect data to must-gather.
2026-09-07T12:45:35.227542586Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-07T12:45:35.237217698Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:35.354860378Z INFO: Waiting for network log collection to complete ...
2026-09-07T12:45:35.355006447Z INFO: Waiting for ovnk database copies to complete ...
2026-09-07T12:45:35.355414177Z INFO: Copying ovnk databases complete.
2026-09-07T12:45:35.357583004Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-07T12:45:35.360100647Z ovnk_database_store/
2026-09-07T12:45:35.360100647Z ovnk_database_store/ovnkube-node-46ffx_nbdb
2026-09-07T12:45:35.371099814Z ovnk_database_store/ovnkube-node-ck4k4_sbdb
2026-09-07T12:45:35.389571749Z ovnk_database_store/ovnkube-node-lmnx5_nbdb
2026-09-07T12:45:35.395986533Z ovnk_database_store/ovnkube-node-lmnx5_sbdb
2026-09-07T12:45:35.413390826Z ovnk_database_store/ovnkube-node-46ffx_sbdb
2026-09-07T12:45:35.431331625Z ovnk_database_store/ovnkube-node-ck4k4_nbdb
2026-09-07T12:45:35.444088160Z INFO: Network log collection complete.
2026-09-07T12:45:35.465387068Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:35.467467062Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-07T12:45:35.472745363Z W0907 12:45:35.472680    1513 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-07T12:45:35.480173986Z Wrote inspect data to must-gather.
2026-09-07T12:45:35.588007874Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:45:35.640816298Z INFO: Getting status from alertmanager-main-0
2026-09-07T12:45:35.688879130Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:35.778189761Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-07T12:45:35.780587476Z INFO: Fallback to identify the container image from release info
2026-09-07T12:45:35.787536508Z Gathering data for ns/openshift-network-console...
2026-09-07T12:45:35.911725284Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:35.914752403Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-07T12:45:36.111617704Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:36.235294856Z Gathering data for ns/openshift-console-operator...
2026-09-07T12:45:36.304356647Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:36.306871393Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-07T12:45:36.454541519Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:36.623669753Z Gathering data for ns/openshift-console...
2026-09-07T12:45:36.635649262Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:45:37.143168107Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-07T12:45:37.277492897Z Done executing Istio gather script
2026-09-07T12:45:37.639828205Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-07T12:45:37.648752574Z Gathering data for ns/openshift-dns-operator...
2026-09-07T12:45:37.766332931Z daemonset.apps/perf-node-gather-daemonset created
2026-09-07T12:45:37.880149580Z Waiting for performance profile collector pods to become ready: 1
2026-09-07T12:45:37.980235030Z Gathering data for ns/openshift-dns...
2026-09-07T12:45:38.003662513Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-07T12:45:38.624498202Z Gathering data for ns/openshift-image-registry...
2026-09-07T12:45:39.001667146Z Waiting for performance profile collector pods to become ready: 2
2026-09-07T12:45:39.123018505Z Gathering data for ns/openshift-ingress-operator...
2026-09-07T12:45:39.474643866Z Gathering data for ns/openshift-ingress...
2026-09-07T12:45:39.814190838Z Gathering data for ns/openshift-ingress-canary...
2026-09-07T12:45:40.111315657Z Waiting for performance profile collector pods to become ready: 3
2026-09-07T12:45:40.219549541Z Gathering data for ns/openshift-insights...
2026-09-07T12:45:41.214003920Z Waiting for performance profile collector pods to become ready: 4
2026-09-07T12:45:42.062414215Z Gathering data for ns/openshift-jobset-operator...
2026-09-07T12:45:42.318419335Z Waiting for performance profile collector pods to become ready: 5
2026-09-07T12:45:42.868328555Z Gathering data for ns/openshift-config...
2026-09-07T12:45:43.008033193Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-07T12:45:43.142791349Z Gathering data for ns/openshift-config-managed...
2026-09-07T12:45:43.430660485Z Waiting for performance profile collector pods to become ready: 6
2026-09-07T12:45:43.577436743Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-07T12:45:43.900827440Z Gathering data for ns/openshift-kube-apiserver...
2026-09-07T12:45:44.185466757Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-07T12:45:44.548832111Z Waiting for performance profile collector pods to become ready: 7
2026-09-07T12:45:44.554687244Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-07T12:45:44.877800983Z Gathering data for ns/openshift-kube-scheduler...
2026-09-07T12:45:45.200665956Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-07T12:45:45.509509164Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-07T12:45:45.664740812Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-07T12:45:45.869569947Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-07T12:45:45.902759545Z Collecting performance related data for node ip-10-0-133-225.ec2.internal
2026-09-07T12:45:45.911741583Z Collecting performance related data for node ip-10-0-137-63.ec2.internal
2026-09-07T12:45:45.923913290Z Collecting performance related data for node ip-10-0-139-210.ec2.internal
2026-09-07T12:45:46.256365240Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-07T12:45:46.980332366Z Gathering data for ns/openshift-multus...
2026-09-07T12:45:48.012680301Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-07T12:45:48.587496257Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-07T12:45:49.857421352Z Gathering data for ns/openshift-host-network...
2026-09-07T12:45:50.187315785Z Gathering data for ns/openshift-network-diagnostics...
2026-09-07T12:45:50.782576333Z Gathering data for ns/openshift-network-node-identity...
2026-09-07T12:45:51.219074900Z Gathering data for ns/openshift-network-operator...
2026-09-07T12:45:51.629901378Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-07T12:45:51.915188177Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-07T12:45:52.423961154Z Gathering data for ns/openshift-apiserver-operator...
2026-09-07T12:45:52.797311176Z Gathering data for ns/openshift-apiserver...
2026-09-07T12:45:53.018937513Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-07T12:45:53.225076413Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-07T12:45:53.512909724Z Gathering data for ns/openshift-controller-manager...
2026-09-07T12:45:53.684030216Z Collecting kubelet logs for node ip-10-0-133-225.ec2.internal
2026-09-07T12:45:53.685698248Z Collecting kubelet logs for node ip-10-0-137-63.ec2.internal
2026-09-07T12:45:53.687860799Z Collecting kubelet logs for node ip-10-0-139-210.ec2.internal
2026-09-07T12:45:53.853404029Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-07T12:45:54.038629271Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5ckf8 namespace
2026-09-07T12:45:54.043422190Z INFO: Node performance data collection complete.
2026-09-07T12:45:54.563925356Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-07T12:45:54.904841478Z Gathering data for ns/openshift-service-ca-operator...
2026-09-07T12:45:55.245817540Z Gathering data for ns/openshift-service-ca...
2026-09-07T12:45:55.621226659Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-07T12:45:56.526547438Z Wrote inspect data to must-gather.
2026-09-07T12:45:56.526588992Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T12:45:56.526588992Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-07T12:45:56.650538422Z error: the server doesn't have a resource type "clusters"
2026-09-07T12:45:56.787673411Z Caches written to disk