2026-09-16T07:34:59.318051163Z [disk usage checker] Started
2026-09-16T07:34:59.321244283Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T07:34:59.501660568Z Gathering data for ns/openshift-cluster-version...
2026-09-16T07:34:59.643104212Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T07:34:59.716534814Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T07:34:59.792912354Z Gathering data for ns/default...
2026-09-16T07:35:00.051258093Z Gathering data for ns/openshift...
2026-09-16T07:35:00.457681083Z Gathering data for ns/kube-system...
2026-09-16T07:35:01.085711676Z Gathering data for ns/openshift-etcd...
2026-09-16T07:35:01.351352872Z Wrote inspect data to must-gather.
2026-09-16T07:35:01.351386501Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T07:35:01.351386501Z     namespaces "assisted-installer" not found
2026-09-16T07:35:01.747441338Z Waiting on subprocesses to finish execution.
2026-09-16T07:35:01.756825745Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T07:35:01.756860625Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T07:35:01.757801979Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T07:35:01.759734991Z Executing Istio gather script
2026-09-16T07:35:01.763898647Z INFO: Gathering HAProxy config files
2026-09-16T07:35:01.831424306Z INFO: Collecting host service logs for crio
2026-09-16T07:35:01.831881758Z INFO: Collecting host service logs for kubelet
2026-09-16T07:35:01.832300458Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T07:35:01.832707851Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T07:35:01.833094722Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T07:35:01.833488373Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T07:35:01.833891162Z INFO: Collecting host service logs for NetworkManager
2026-09-16T07:35:01.834263364Z INFO: Collecting host service logs for openvswitch
2026-09-16T07:35:01.834662000Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T07:35:01.835023620Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T07:35:01.835374685Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T07:35:01.835796545Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T07:35:01.844808910Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T07:35:02.260973415Z No resources found in openshift-etcd namespace.
2026-09-16T07:35:02.621757416Z INFO: Collecting Insights Archives from insights-operator-5844b856db-lbfds
2026-09-16T07:35:02.621757416Z insights-runtime-extractor-62gdl
2026-09-16T07:35:02.621757416Z insights-runtime-extractor-72z5r
2026-09-16T07:35:02.621757416Z insights-runtime-extractor-c42bs
2026-09-16T07:35:02.677596388Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T07:35:02.712355999Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T07:35:02.718788165Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T07:35:02.735479220Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T07:35:02.735618182Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T07:35:02.879747337Z No resources found
2026-09-16T07:35:02.968784382Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T07:35:02.971122696Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T07:35:03.055917734Z Wrote inspect data to must-gather.
2026-09-16T07:35:03.078153988Z Wrote inspect data to must-gather.
2026-09-16T07:35:03.235151036Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T07:35:03.235151036Z See 'oc rsync -h' for help and examples
2026-09-16T07:35:03.242807149Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T07:35:03.281295572Z INFO: Worker host service log collection to complete.
2026-09-16T07:35:03.362368596Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T07:35:03.421439302Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T07:35:03.479364459Z ERROR: No running kube-apiserver pods found
2026-09-16T07:35:03.503314816Z Wrote inspect data to must-gather.
2026-09-16T07:35:03.685750960Z tar: Removing leading `/' from member names
2026-09-16T07:35:03.698233703Z INFO: HAProxy config collection complete.
2026-09-16T07:35:04.026741813Z No resources found
2026-09-16T07:35:04.065841874Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T07:35:04.071909023Z Wrote inspect data to must-gather.
2026-09-16T07:35:04.074653843Z Gathering data for ns/openshift-monitoring...
2026-09-16T07:35:04.117666707Z Wrote inspect data to must-gather.
2026-09-16T07:35:04.163967652Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T07:35:04.184938036Z INFO: 'previous-logs' folder not found on ip-10-0-131-70.ec2.internal, skipping...
2026-09-16T07:35:04.254236868Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T07:35:04.322331427Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T07:35:04.348911723Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T07:35:04.350429659Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T07:35:04.368077411Z Wrote inspect data to must-gather.
2026-09-16T07:35:04.429717314Z Wrote inspect data to must-gather.
2026-09-16T07:35:04.515957704Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T07:35:04.561253427Z INFO: 'previous-logs' folder not found on ip-10-0-137-143.ec2.internal, skipping...
2026-09-16T07:35:04.611742036Z Wrote inspect data to must-gather.
2026-09-16T07:35:04.731181194Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T07:35:04.763411263Z Wrote inspect data to must-gather.
2026-09-16T07:35:04.828669470Z INFO: 'previous-logs' folder not found on ip-10-0-138-88.ec2.internal, skipping...
2026-09-16T07:35:04.828741279Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T07:35:04.828787664Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T07:35:04.859209603Z Wrote inspect data to must-gather.
2026-09-16T07:35:04.917821871Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T07:35:05.088289686Z Wrote inspect data to must-gather.
2026-09-16T07:35:05.122785774Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T07:35:05.136979949Z Wrote inspect data to must-gather.
2026-09-16T07:35:05.213306957Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T07:35:05.337338971Z Wrote inspect data to must-gather.
2026-09-16T07:35:05.339890586Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T07:35:05.368219845Z INFO: INTERCONNECT MODE
2026-09-16T07:35:05.368253523Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T07:35:05.393086913Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:05.548868522Z INFO: Gathering OVN_Northbound from ovnkube-node-8zdhq...
2026-09-16T07:35:05.549121945Z INFO: Gathering OVN_Northbound from ovnkube-node-mp64j...
2026-09-16T07:35:05.549336959Z INFO: Gathering OVN_Northbound from ovnkube-node-p7hh7...
2026-09-16T07:35:05.549678585Z INFO: Gathering OVN_Southbound from ovnkube-node-8zdhq...
2026-09-16T07:35:05.549961321Z INFO: Gathering OVN_Southbound from ovnkube-node-mp64j...
2026-09-16T07:35:05.550251510Z INFO: Gathering OVN_Southbound from ovnkube-node-p7hh7...
2026-09-16T07:35:05.565968717Z Wrote inspect data to must-gather.
2026-09-16T07:35:05.591003775Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T07:35:05.672013988Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:05.678108228Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T07:35:05.755768275Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T07:35:05.818876501Z tar: Removing leading `/' from member names
2026-09-16T07:35:05.823772994Z tar: Removing leading `/' from member names
2026-09-16T07:35:05.828765951Z tar: Removing leading `/' from member names
2026-09-16T07:35:05.836811626Z tar: Removing leading `/' from member names
2026-09-16T07:35:05.895880929Z tar: Removing leading `/' from member names
2026-09-16T07:35:05.901796326Z tar: Removing leading `/' from member names
2026-09-16T07:35:05.918926056Z Wrote inspect data to must-gather.
2026-09-16T07:35:05.945248626Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T07:35:05.981544595Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:06.157790793Z Wrote inspect data to must-gather.
2026-09-16T07:35:06.229503911Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:06.232722087Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T07:35:06.367785916Z INFO: Getting status from alertmanager-main-0
2026-09-16T07:35:06.374106411Z INFO: Waiting for network log collection to complete ...
2026-09-16T07:35:06.374160177Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T07:35:06.374694476Z INFO: Copying ovnk databases complete.
2026-09-16T07:35:06.376440673Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-09-16T07:35:06.380115966Z ovnk_database_store/
2026-09-16T07:35:06.380115966Z ovnk_database_store/ovnkube-node-p7hh7_nbdb
2026-09-16T07:35:06.386794914Z ovnk_database_store/ovnkube-node-8zdhq_sbdb
2026-09-16T07:35:06.390193757Z W0916 07:35:06.390153    1495 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-16T07:35:06.397257863Z Wrote inspect data to must-gather.
2026-09-16T07:35:06.409328413Z ovnk_database_store/ovnkube-node-p7hh7_sbdb
2026-09-16T07:35:06.426863948Z ovnk_database_store/ovnkube-node-mp64j_nbdb
2026-09-16T07:35:06.438513934Z ovnk_database_store/ovnkube-node-mp64j_sbdb
2026-09-16T07:35:06.453660453Z ovnk_database_store/ovnkube-node-8zdhq_nbdb
2026-09-16T07:35:06.458956796Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:06.462915845Z INFO: Network log collection complete.
2026-09-16T07:35:06.471062173Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T07:35:06.648835212Z Gathering data for ns/openshift-network-console...
2026-09-16T07:35:06.661538668Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:06.663679668Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T07:35:06.682417544Z Wrote inspect data to must-gather.
2026-09-16T07:35:06.871415841Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:06.935388980Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T07:35:06.938002699Z INFO: Fallback to identify the container image from release info
2026-09-16T07:35:07.048496481Z Gathering data for ns/openshift-console-operator...
2026-09-16T07:35:07.069521861Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:07.071975178Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T07:35:07.224935614Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:07.398517260Z Gathering data for ns/openshift-console...
2026-09-16T07:35:07.403140101Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:35:07.533838012Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T07:35:07.651845552Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T07:35:07.781762957Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T07:35:07.915812464Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T07:35:08.079485579Z Done executing Istio gather script
2026-09-16T07:35:08.378446513Z Gathering data for ns/openshift-dns-operator...
2026-09-16T07:35:08.659187396Z Gathering data for ns/openshift-dns...
2026-09-16T07:35:08.889912346Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T07:35:09.184445456Z Gathering data for ns/openshift-image-registry...
2026-09-16T07:35:09.357886170Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T07:35:09.636893628Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T07:35:09.965263654Z Gathering data for ns/openshift-ingress...
2026-09-16T07:35:10.004872615Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T07:35:10.286828068Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T07:35:10.635481333Z Gathering data for ns/openshift-insights...
2026-09-16T07:35:11.118882578Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T07:35:12.236393339Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T07:35:12.382180331Z Gathering data for ns/openshift-config...
2026-09-16T07:35:12.603644645Z Gathering data for ns/openshift-config-managed...
2026-09-16T07:35:12.978952868Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T07:35:13.244337581Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T07:35:13.352546245Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T07:35:13.482862704Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T07:35:13.748252818Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T07:35:14.009074484Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T07:35:14.229818376Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T07:35:14.363015917Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T07:35:14.466313734Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T07:35:14.505572240Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T07:35:14.794313638Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T07:35:15.069727018Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T07:35:15.504443018Z Gathering data for ns/openshift-multus...
2026-09-16T07:35:15.572575876Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T07:35:15.778678094Z Collecting performance related data for node ip-10-0-138-88.ec2.internal
2026-09-16T07:35:15.785393525Z Collecting performance related data for node ip-10-0-137-143.ec2.internal
2026-09-16T07:35:15.792786692Z Collecting performance related data for node ip-10-0-131-70.ec2.internal
2026-09-16T07:35:17.011706350Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T07:35:18.397393158Z Gathering data for ns/openshift-host-network...
2026-09-16T07:35:18.695272918Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T07:35:19.195369147Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T07:35:19.368238029Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T07:35:19.588983878Z Gathering data for ns/openshift-network-operator...
2026-09-16T07:35:19.946507056Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T07:35:20.191682803Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T07:35:20.871763990Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T07:35:21.134712902Z Gathering data for ns/openshift-apiserver...
2026-09-16T07:35:21.500650403Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T07:35:21.731354134Z Gathering data for ns/openshift-controller-manager...
2026-09-16T07:35:21.975210316Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T07:35:22.607349255Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T07:35:22.882835313Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T07:35:23.170271264Z Gathering data for ns/openshift-service-ca...
2026-09-16T07:35:23.506260280Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T07:35:23.560208143Z Collecting kubelet logs for node ip-10-0-138-88.ec2.internal
2026-09-16T07:35:23.561965066Z Collecting kubelet logs for node ip-10-0-137-143.ec2.internal
2026-09-16T07:35:23.563592536Z Collecting kubelet logs for node ip-10-0-131-70.ec2.internal
2026-09-16T07:35:23.872008194Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-sqf6p namespace
2026-09-16T07:35:23.876580888Z INFO: Node performance data collection complete.
2026-09-16T07:35:24.372804176Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T07:35:24.421467210Z Wrote inspect data to must-gather.
2026-09-16T07:35:24.421510309Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T07:35:24.421510309Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T07:35:24.539752787Z error: the server doesn't have a resource type "clusters"
2026-09-16T07:35:24.663872519Z Caches written to disk