2026-08-03T15:05:34.417839941Z [disk usage checker] Started
2026-08-03T15:05:34.420951110Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-03T15:05:34.604924779Z Gathering data for ns/openshift-cluster-version...
2026-08-03T15:05:34.739989677Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-03T15:05:34.824437753Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-03T15:05:34.903014324Z Gathering data for ns/default...
2026-08-03T15:05:35.169365287Z Gathering data for ns/openshift...
2026-08-03T15:05:35.544991178Z Gathering data for ns/kube-system...
2026-08-03T15:05:36.325517270Z Gathering data for ns/openshift-etcd...
2026-08-03T15:05:36.696393770Z Wrote inspect data to must-gather.
2026-08-03T15:05:36.696427687Z error: inspection completed with the errors occurred while gathering data:
2026-08-03T15:05:36.696427687Z     namespaces "assisted-installer" not found
2026-08-03T15:05:36.916007687Z Waiting on subprocesses to finish execution.
2026-08-03T15:05:36.929019628Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-08-03T15:05:36.933324533Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-08-03T15:05:36.936252215Z INFO: Gathering HAProxy config files
2026-08-03T15:05:36.936584892Z INFO: Gathering machine config daemon's old logs from all nodes
2026-08-03T15:05:36.936699133Z INFO: Collecting host service logs for crio
2026-08-03T15:05:36.937076169Z INFO: Collecting host service logs for kubelet
2026-08-03T15:05:36.937441339Z INFO: Collecting host service logs for rpm-ostreed
2026-08-03T15:05:36.937730934Z INFO: Collecting host service logs for ostree-finalize-staged
2026-08-03T15:05:36.938037115Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-08-03T15:05:36.938346812Z INFO: Collecting host service logs for machine-config-daemon-host
2026-08-03T15:05:36.938631927Z INFO: Collecting host service logs for NetworkManager
2026-08-03T15:05:36.938933001Z INFO: Collecting host service logs for openvswitch
2026-08-03T15:05:36.939308465Z INFO: Collecting host service logs for ovs-configuration
2026-08-03T15:05:36.939607480Z INFO: Collecting host service logs for ovsdb-server
2026-08-03T15:05:36.939909545Z INFO: Collecting host service logs for ovs-vswitchd
2026-08-03T15:05:36.940300828Z INFO: Waiting for worker host service log collection to complete ...
2026-08-03T15:05:36.955086653Z Executing Istio gather script
2026-08-03T15:05:37.196210711Z INFO: Waiting for node performance related collection to complete ...
2026-08-03T15:05:37.462792021Z Wrote inspect data to must-gather.
2026-08-03T15:05:37.571095417Z INFO: "metallb-operator" not detected. Skipping.
2026-08-03T15:05:37.611698782Z No resources found in openshift-etcd namespace.
2026-08-03T15:05:37.641860034Z INFO: Collecting Insights Archives from insights-operator-77696685b-mw67c
2026-08-03T15:05:37.641860034Z insights-runtime-extractor-d4b8l
2026-08-03T15:05:37.641860034Z insights-runtime-extractor-nvp8b
2026-08-03T15:05:37.641860034Z insights-runtime-extractor-s4w4b
2026-08-03T15:05:37.694366982Z INFO: Found 1 replicas - prometheus-k8s-0
2026-08-03T15:05:37.759080301Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-08-03T15:05:37.885722779Z No resources found
2026-08-03T15:05:37.929352454Z INFO: "sriov-network-operator" not detected. Skipping.
2026-08-03T15:05:38.179612094Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-08-03T15:05:38.179612094Z See 'oc rsync -h' for help and examples
2026-08-03T15:05:38.267430476Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-08-03T15:05:38.378885943Z ERROR: No running kube-apiserver pods found
2026-08-03T15:05:38.530857903Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-03T15:05:38.653747483Z No resources found
2026-08-03T15:05:38.724220388Z Wrote inspect data to must-gather.
2026-08-03T15:05:38.858207182Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-08-03T15:05:38.881332349Z error: the server doesn't have a resource type "performanceprofile"
2026-08-03T15:05:38.988111973Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-08-03T15:05:38.988184571Z INFO: on-disk MachineConfig config collection complete.
2026-08-03T15:05:38.992939114Z INFO: Worker host service log collection to complete.
2026-08-03T15:05:39.008110236Z INFO: Waiting for HAProxy config collection to complete ...
2026-08-03T15:05:39.031752826Z Wrote inspect data to must-gather.
2026-08-03T15:05:39.199198524Z Wrote inspect data to must-gather.
2026-08-03T15:05:39.209517776Z Wrote inspect data to must-gather.
2026-08-03T15:05:39.213421699Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-08-03T15:05:39.241829594Z INFO: 'previous-logs' folder not found on ip-10-0-130-162.ec2.internal, skipping...
2026-08-03T15:05:39.332716410Z Wrote inspect data to must-gather.
2026-08-03T15:05:39.455429444Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-03T15:05:39.463732083Z tar: Removing leading `/' from member names
2026-08-03T15:05:39.467959955Z INFO: HAProxy config collection complete.
2026-08-03T15:05:39.505584611Z INFO: Getting rules from prometheus-k8s-0
2026-08-03T15:05:39.553517962Z Wrote inspect data to must-gather.
2026-08-03T15:05:39.624517523Z INFO: 'previous-logs' folder not found on ip-10-0-133-58.ec2.internal, skipping...
2026-08-03T15:05:39.672357137Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-08-03T15:05:39.683285333Z Wrote inspect data to must-gather.
2026-08-03T15:05:39.748837923Z error: the server doesn't have a resource type "machineconfigs"
2026-08-03T15:05:39.893119640Z Gathering data for ns/openshift-monitoring...
2026-08-03T15:05:39.914976543Z INFO: 'previous-logs' folder not found on ip-10-0-137-187.ec2.internal, skipping...
2026-08-03T15:05:39.915100001Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-08-03T15:05:39.915155294Z INFO: Machine Config Daemon termination log collection complete.
2026-08-03T15:05:39.926472659Z error: the server doesn't have a resource type "machineconfigpools"
2026-08-03T15:05:39.961166932Z INFO: Getting status/config from prometheus-k8s-0
2026-08-03T15:05:39.967636127Z Wrote inspect data to must-gather.
2026-08-03T15:05:40.149716711Z Wrote inspect data to must-gather.
2026-08-03T15:05:40.207866843Z Wrote inspect data to must-gather.
2026-08-03T15:05:40.307371029Z error: the server doesn't have a resource type "kubeletconfigs"
2026-08-03T15:05:40.347191479Z INFO: Getting status/flags from prometheus-k8s-0
2026-08-03T15:05:40.492789107Z INFO: INTERCONNECT MODE
2026-08-03T15:05:40.492818914Z INFO: Gathering ovn-kubernetes DBs
2026-08-03T15:05:40.529151796Z Wrote inspect data to must-gather.
2026-08-03T15:05:40.579864118Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-08-03T15:05:40.586427254Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-08-03T15:05:40.647109482Z INFO: Gathering OVN_Northbound from ovnkube-node-4jd5s...
2026-08-03T15:05:40.647370068Z INFO: Gathering OVN_Northbound from ovnkube-node-bwk7s...
2026-08-03T15:05:40.647661011Z INFO: Gathering OVN_Northbound from ovnkube-node-wqq6m...
2026-08-03T15:05:40.647958634Z INFO: Gathering OVN_Southbound from ovnkube-node-4jd5s...
2026-08-03T15:05:40.648250069Z INFO: Gathering OVN_Southbound from ovnkube-node-bwk7s...
2026-08-03T15:05:40.648573173Z INFO: Gathering OVN_Southbound from ovnkube-node-wqq6m...
2026-08-03T15:05:40.798711336Z Wrote inspect data to must-gather.
2026-08-03T15:05:40.849219080Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:40.867718921Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-08-03T15:05:40.921064859Z tar: Removing leading `/' from member names
2026-08-03T15:05:40.937471186Z tar: Removing leading `/' from member names
2026-08-03T15:05:40.939317558Z tar: Removing leading `/' from member names
2026-08-03T15:05:40.989174234Z tar: Removing leading `/' from member names
2026-08-03T15:05:40.999190835Z tar: Removing leading `/' from member names
2026-08-03T15:05:41.002438616Z tar: Removing leading `/' from member names
2026-08-03T15:05:41.080407415Z Wrote inspect data to must-gather.
2026-08-03T15:05:41.104352232Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:41.106995577Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-08-03T15:05:41.175053198Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-08-03T15:05:41.311385563Z Wrote inspect data to must-gather.
2026-08-03T15:05:41.329735819Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:41.433851042Z INFO: Waiting for network log collection to complete ...
2026-08-03T15:05:41.433904229Z INFO: Waiting for ovnk database copies to complete ...
2026-08-03T15:05:41.434431812Z INFO: Copying ovnk databases complete.
2026-08-03T15:05:41.436215604Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-08-03T15:05:41.440023353Z ovnk_database_store/
2026-08-03T15:05:41.440023353Z ovnk_database_store/ovnkube-node-wqq6m_sbdb
2026-08-03T15:05:41.462760803Z ovnk_database_store/ovnkube-node-4jd5s_sbdb
2026-08-03T15:05:41.481329331Z ovnk_database_store/ovnkube-node-wqq6m_nbdb
2026-08-03T15:05:41.487529980Z ovnk_database_store/ovnkube-node-bwk7s_sbdb
2026-08-03T15:05:41.498874128Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-03T15:05:41.507649150Z ovnk_database_store/ovnkube-node-4jd5s_nbdb
2026-08-03T15:05:41.514964154Z ovnk_database_store/ovnkube-node-bwk7s_nbdb
2026-08-03T15:05:41.523429096Z INFO: Network log collection complete.
2026-08-03T15:05:41.558521818Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:41.562678582Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-08-03T15:05:41.568361996Z Wrote inspect data to must-gather.
2026-08-03T15:05:41.591253416Z INFO: Getting status from alertmanager-main-0
2026-08-03T15:05:41.762640571Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:41.772799000Z Wrote inspect data to must-gather.
2026-08-03T15:05:41.945707488Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:41.949174067Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-08-03T15:05:41.965022941Z W0803 15:05:41.964975    1576 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-08-03T15:05:41.971970211Z Wrote inspect data to must-gather.
2026-08-03T15:05:42.083124694Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-03T15:05:42.170337283Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:42.231581039Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-08-03T15:05:42.234526966Z INFO: Fallback to identify the container image from release info
2026-08-03T15:05:42.253411895Z Gathering data for ns/openshift-network-console...
2026-08-03T15:05:42.372103886Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:42.374300964Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-08-03T15:05:42.525328377Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:42.656075136Z Gathering data for ns/openshift-console-operator...
2026-08-03T15:05:42.719296038Z Wrote inspect data to /must-gather/istio.
2026-08-03T15:05:42.841282032Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-08-03T15:05:42.965485746Z daemonset.apps/perf-node-gather-daemonset created
2026-08-03T15:05:43.050474329Z Gathering data for ns/openshift-console...
2026-08-03T15:05:43.095131926Z Waiting for performance profile collector pods to become ready: 1
2026-08-03T15:05:43.399839273Z Done executing Istio gather script
2026-08-03T15:05:43.549047334Z Gathering data for ns/openshift-cluster-storage-operator...
2026-08-03T15:05:44.009516996Z Gathering data for ns/openshift-dns-operator...
2026-08-03T15:05:44.214153518Z Waiting for performance profile collector pods to become ready: 2
2026-08-03T15:05:44.282364527Z Gathering data for ns/openshift-dns...
2026-08-03T15:05:44.463851060Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-03T15:05:44.932655540Z Gathering data for ns/openshift-image-registry...
2026-08-03T15:05:45.337848446Z Waiting for performance profile collector pods to become ready: 3
2026-08-03T15:05:45.424609306Z Gathering data for ns/openshift-ingress-operator...
2026-08-03T15:05:45.727209004Z Gathering data for ns/openshift-ingress...
2026-08-03T15:05:46.050897843Z Gathering data for ns/openshift-ingress-canary...
2026-08-03T15:05:46.422140494Z Gathering data for ns/openshift-insights...
2026-08-03T15:05:46.454657576Z Waiting for performance profile collector pods to become ready: 4
2026-08-03T15:05:47.559720258Z Waiting for performance profile collector pods to become ready: 5
2026-08-03T15:05:48.312993347Z Gathering data for ns/openshift-jobset-operator...
2026-08-03T15:05:48.676583097Z Waiting for performance profile collector pods to become ready: 6
2026-08-03T15:05:49.071436830Z Gathering data for ns/openshift-config...
2026-08-03T15:05:49.350276117Z Gathering data for ns/openshift-config-managed...
2026-08-03T15:05:49.468138497Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-03T15:05:49.745163440Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-08-03T15:05:49.795605986Z Waiting for performance profile collector pods to become ready: 7
2026-08-03T15:05:49.996243109Z Gathering data for ns/openshift-kube-apiserver...
2026-08-03T15:05:50.242978364Z Gathering data for ns/openshift-kube-controller-manager...
2026-08-03T15:05:50.551292789Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-08-03T15:05:50.818261606Z Gathering data for ns/openshift-kube-scheduler...
2026-08-03T15:05:50.915492295Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-08-03T15:05:51.086831722Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-08-03T15:05:51.155365745Z Collecting performance related data for node ip-10-0-133-58.ec2.internal
2026-08-03T15:05:51.163353604Z Collecting performance related data for node ip-10-0-137-187.ec2.internal
2026-08-03T15:05:51.174781186Z Collecting performance related data for node ip-10-0-130-162.ec2.internal
2026-08-03T15:05:51.425315017Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-08-03T15:05:51.800329982Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-08-03T15:05:52.180146274Z Gathering data for ns/openshift-user-workload-monitoring...
2026-08-03T15:05:52.765896818Z Gathering data for ns/openshift-multus...
2026-08-03T15:05:54.227393930Z Gathering data for ns/openshift-ovn-kubernetes...
2026-08-03T15:05:54.472839503Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-03T15:05:55.497180008Z Gathering data for ns/openshift-host-network...
2026-08-03T15:05:55.793231306Z Gathering data for ns/openshift-network-diagnostics...
2026-08-03T15:05:56.320173711Z Gathering data for ns/openshift-network-node-identity...
2026-08-03T15:05:56.699912026Z Gathering data for ns/openshift-network-operator...
2026-08-03T15:05:57.074287285Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-08-03T15:05:57.384984721Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-08-03T15:05:57.851853068Z Gathering data for ns/openshift-apiserver-operator...
2026-08-03T15:05:58.177386600Z Gathering data for ns/openshift-apiserver...
2026-08-03T15:05:58.535752917Z Gathering data for ns/openshift-controller-manager-operator...
2026-08-03T15:05:58.803627272Z Gathering data for ns/openshift-controller-manager...
2026-08-03T15:05:59.049722952Z Collecting kubelet logs for node ip-10-0-133-58.ec2.internal
2026-08-03T15:05:59.051402290Z Collecting kubelet logs for node ip-10-0-137-187.ec2.internal
2026-08-03T15:05:59.053134202Z Collecting kubelet logs for node ip-10-0-130-162.ec2.internal
2026-08-03T15:05:59.093372219Z Gathering data for ns/openshift-cluster-samples-operator...
2026-08-03T15:05:59.373207960Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-tjzs8 namespace
2026-08-03T15:05:59.377860773Z INFO: Node performance data collection complete.
2026-08-03T15:05:59.477245634Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-03T15:05:59.774096442Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-08-03T15:06:00.046576007Z Gathering data for ns/openshift-service-ca-operator...
2026-08-03T15:06:00.340357548Z Gathering data for ns/openshift-service-ca...
2026-08-03T15:06:00.691743417Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-08-03T15:06:01.562773046Z Wrote inspect data to must-gather.
2026-08-03T15:06:01.562817611Z error: inspection completed with the errors occurred while gathering data:
2026-08-03T15:06:01.562817611Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-08-03T15:06:01.684128884Z error: the server doesn't have a resource type "clusters"
2026-08-03T15:06:01.812303442Z Caches written to disk