2026-09-16T19:40:55.465910855Z [disk usage checker] Started
2026-09-16T19:40:55.469123494Z [disk usage checker] Volume usage percentage: current = 34 ; allowed = 70
2026-09-16T19:40:55.848395701Z Gathering data for ns/openshift-cluster-version...
2026-09-16T19:40:55.991127998Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:40:56.085663370Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:40:56.170489080Z Gathering data for ns/default...
2026-09-16T19:40:56.462549578Z Gathering data for ns/openshift...
2026-09-16T19:40:56.872604117Z Gathering data for ns/kube-system...
2026-09-16T19:40:57.606927427Z Gathering data for ns/openshift-etcd...
2026-09-16T19:40:57.934659682Z Wrote inspect data to must-gather.
2026-09-16T19:40:57.934694995Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:40:57.934694995Z     namespaces "assisted-installer" not found
2026-09-16T19:40:58.532034209Z Waiting on subprocesses to finish execution.
2026-09-16T19:40:58.540873842Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T19:40:58.545149832Z Executing Istio gather script
2026-09-16T19:40:58.549662934Z INFO: Gathering HAProxy config files
2026-09-16T19:40:58.550503751Z INFO: Collecting host service logs for crio
2026-09-16T19:40:58.550916264Z INFO: Collecting host service logs for kubelet
2026-09-16T19:40:58.551321161Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T19:40:58.551660817Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T19:40:58.552139121Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T19:40:58.552520332Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T19:40:58.552875004Z INFO: Collecting host service logs for NetworkManager
2026-09-16T19:40:58.553644848Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T19:40:58.554476637Z INFO: Collecting host service logs for openvswitch
2026-09-16T19:40:58.554864982Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T19:40:58.555089175Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T19:40:58.555261127Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T19:40:58.555581110Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T19:40:58.555970968Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T19:40:58.878582509Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T19:40:59.249668734Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T19:40:59.348611366Z Wrote inspect data to must-gather.
2026-09-16T19:40:59.746413535Z No resources found in openshift-etcd namespace.
2026-09-16T19:40:59.801640417Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T19:40:59.941175362Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T19:41:00.022587766Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T19:41:00.094999543Z INFO: Collecting Insights Archives from insights-operator-5844b856db-h97rl
2026-09-16T19:41:00.095059682Z insights-runtime-extractor-2dtv9
2026-09-16T19:41:00.095078114Z insights-runtime-extractor-nrxcw
2026-09-16T19:41:00.095095237Z insights-runtime-extractor-tdz2g
2026-09-16T19:41:00.142273163Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T19:41:00.251314387Z No resources found
2026-09-16T19:41:00.546619103Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T19:41:00.551805520Z Wrote inspect data to must-gather.
2026-09-16T19:41:00.578332936Z [disk usage checker] Volume usage percentage: current = 34 ; allowed = 70
2026-09-16T19:41:00.613395449Z No resources found
2026-09-16T19:41:00.654397458Z INFO: Worker host service log collection to complete.
2026-09-16T19:41:00.771576159Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T19:41:00.817954147Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T19:41:00.818030623Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T19:41:00.934568615Z Wrote inspect data to must-gather.
2026-09-16T19:41:00.937888369Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:41:00.963892721Z Gathering data for ns/openshift-monitoring...
2026-09-16T19:41:00.998697697Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T19:41:01.066139404Z Wrote inspect data to must-gather.
2026-09-16T19:41:01.068033630Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T19:41:01.068033630Z See 'oc rsync -h' for help and examples
2026-09-16T19:41:01.099586865Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T19:41:01.197290967Z tar: Removing leading `/' from member names
2026-09-16T19:41:01.205314000Z INFO: HAProxy config collection complete.
2026-09-16T19:41:01.329400330Z Wrote inspect data to must-gather.
2026-09-16T19:41:01.364690985Z ERROR: No running kube-apiserver pods found
2026-09-16T19:41:01.424823927Z Wrote inspect data to must-gather.
2026-09-16T19:41:01.446247374Z Wrote inspect data to must-gather.
2026-09-16T19:41:01.473974374Z INFO: 'previous-logs' folder not found on ip-10-0-137-241.ec2.internal, skipping...
2026-09-16T19:41:01.619222322Z Wrote inspect data to must-gather.
2026-09-16T19:41:01.650320221Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T19:41:01.789489410Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T19:41:01.806985004Z INFO: 'previous-logs' folder not found on ip-10-0-137-37.ec2.internal, skipping...
2026-09-16T19:41:01.852943250Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T19:41:01.955688861Z Wrote inspect data to must-gather.
2026-09-16T19:41:02.139931505Z INFO: 'previous-logs' folder not found on ip-10-0-139-120.ec2.internal, skipping...
2026-09-16T19:41:02.139984387Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T19:41:02.140018643Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T19:41:02.153261145Z Wrote inspect data to must-gather.
2026-09-16T19:41:02.279368009Z Wrote inspect data to must-gather.
2026-09-16T19:41:02.366898181Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T19:41:02.375886097Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T19:41:02.636366942Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T19:41:02.641711798Z Wrote inspect data to must-gather.
2026-09-16T19:41:02.703670991Z INFO: INTERCONNECT MODE
2026-09-16T19:41:02.703782111Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T19:41:02.850072374Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:41:02.889489684Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T19:41:02.890980212Z INFO: Gathering OVN_Northbound from ovnkube-node-kdln4...
2026-09-16T19:41:02.891263407Z INFO: Gathering OVN_Northbound from ovnkube-node-lpxk7...
2026-09-16T19:41:02.891607712Z INFO: Gathering OVN_Northbound from ovnkube-node-pxcjq...
2026-09-16T19:41:02.891952719Z INFO: Gathering OVN_Southbound from ovnkube-node-kdln4...
2026-09-16T19:41:02.892203221Z INFO: Gathering OVN_Southbound from ovnkube-node-lpxk7...
2026-09-16T19:41:02.892474116Z INFO: Gathering OVN_Southbound from ovnkube-node-pxcjq...
2026-09-16T19:41:02.907057711Z Wrote inspect data to must-gather.
2026-09-16T19:41:03.278900795Z tar: Removing leading `/' from member names
2026-09-16T19:41:03.279302032Z tar: Removing leading `/' from member names
2026-09-16T19:41:03.297980502Z tar: Removing leading `/' from member names
2026-09-16T19:41:03.323121371Z tar: Removing leading `/' from member names
2026-09-16T19:41:03.329937718Z tar: Removing leading `/' from member names
2026-09-16T19:41:03.389982547Z tar: Removing leading `/' from member names
2026-09-16T19:41:03.392143361Z Wrote inspect data to must-gather.
2026-09-16T19:41:03.451824332Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T19:41:03.633301299Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T19:41:03.671904277Z W0916 19:41:03.671852    1386 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-16T19:41:03.682678355Z Wrote inspect data to must-gather.
2026-09-16T19:41:03.715073387Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:41:03.765396398Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T19:41:03.880472595Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:03.940669420Z INFO: Waiting for network log collection to complete ...
2026-09-16T19:41:03.940789494Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T19:41:03.941230232Z INFO: Copying ovnk databases complete.
2026-09-16T19:41:03.946943038Z 4.8M	must-gather/network_logs/ovnk_database_store
2026-09-16T19:41:03.951635477Z ovnk_database_store/
2026-09-16T19:41:03.951635477Z ovnk_database_store/ovnkube-node-lpxk7_nbdb
2026-09-16T19:41:03.962843498Z Gathering data for ns/openshift-network-console...
2026-09-16T19:41:03.963819226Z ovnk_database_store/ovnkube-node-lpxk7_sbdb
2026-09-16T19:41:03.994013958Z ovnk_database_store/ovnkube-node-kdln4_sbdb
2026-09-16T19:41:04.025803744Z ovnk_database_store/ovnkube-node-pxcjq_nbdb
2026-09-16T19:41:04.036769354Z ovnk_database_store/ovnkube-node-kdln4_nbdb
2026-09-16T19:41:04.051182655Z ovnk_database_store/ovnkube-node-pxcjq_sbdb
2026-09-16T19:41:04.069404459Z Wrote inspect data to must-gather.
2026-09-16T19:41:04.082288836Z INFO: Network log collection complete.
2026-09-16T19:41:04.137883890Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:04.141748389Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T19:41:04.144425364Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T19:41:04.383713964Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:04.426197175Z Wrote inspect data to must-gather.
2026-09-16T19:41:04.500244035Z Gathering data for ns/openshift-console-operator...
2026-09-16T19:41:04.609829762Z INFO: Getting status from alertmanager-main-0
2026-09-16T19:41:04.640327472Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:04.644544302Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T19:41:04.768889573Z Wrote inspect data to must-gather.
2026-09-16T19:41:04.872791028Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:04.975221985Z Gathering data for ns/openshift-console...
2026-09-16T19:41:05.101891471Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T19:41:05.105759034Z INFO: Fallback to identify the container image from release info
2026-09-16T19:41:05.106713007Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:05.109734368Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T19:41:05.330601535Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:05.556192712Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:05.560707773Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-16T19:41:05.587964744Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T19:41:05.602491477Z [disk usage checker] Volume usage percentage: current = 34 ; allowed = 70
2026-09-16T19:41:05.702019994Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T19:41:05.762203749Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:05.841561779Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T19:41:06.000288333Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:06.003899914Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T19:41:06.005580561Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-16T19:41:06.170094959Z Gathering data for ns/openshift-dns-operator...
2026-09-16T19:41:06.200048728Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:06.434621913Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:06.438055052Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T19:41:06.512959092Z Gathering data for ns/openshift-dns...
2026-09-16T19:41:06.619875286Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:06.817940641Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:41:07.164077224Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T19:41:07.182415874Z Gathering data for ns/openshift-image-registry...
2026-09-16T19:41:07.559320152Z Done executing Istio gather script
2026-09-16T19:41:07.677317736Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T19:41:08.056991724Z Gathering data for ns/openshift-ingress...
2026-09-16T19:41:08.287807376Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T19:41:08.408493210Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T19:41:08.808415689Z Gathering data for ns/openshift-insights...
2026-09-16T19:41:09.422317986Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T19:41:10.553648937Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T19:41:10.607382628Z [disk usage checker] Volume usage percentage: current = 34 ; allowed = 70
2026-09-16T19:41:11.257808208Z Gathering data for ns/kserve...
2026-09-16T19:41:11.705465072Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T19:41:12.839904457Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T19:41:13.372869687Z Gathering data for ns/openshift-config...
2026-09-16T19:41:13.676595353Z Gathering data for ns/openshift-config-managed...
2026-09-16T19:41:13.960302909Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T19:41:14.160358756Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T19:41:14.248103355Z Collecting performance related data for node ip-10-0-137-241.ec2.internal
2026-09-16T19:41:14.255195641Z Collecting performance related data for node ip-10-0-137-37.ec2.internal
2026-09-16T19:41:14.262311050Z Collecting performance related data for node ip-10-0-139-120.ec2.internal
2026-09-16T19:41:14.478931161Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T19:41:14.831084962Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T19:41:15.208329618Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T19:41:15.614696323Z [disk usage checker] Volume usage percentage: current = 34 ; allowed = 70
2026-09-16T19:41:15.642798781Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T19:41:15.987746406Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T19:41:16.291614941Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T19:41:16.645196533Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T19:41:17.004950463Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T19:41:17.615223073Z Gathering data for ns/openshift-multus...
2026-09-16T19:41:19.122520176Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T19:41:20.445879896Z Gathering data for ns/openshift-host-network...
2026-09-16T19:41:20.620274029Z [disk usage checker] Volume usage percentage: current = 34 ; allowed = 70
2026-09-16T19:41:20.823960245Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T19:41:21.390548217Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T19:41:21.828593599Z Gathering data for ns/openshift-network-operator...
2026-09-16T19:41:22.202959108Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T19:41:22.475051372Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T19:41:22.960536087Z Collecting kubelet logs for node ip-10-0-137-241.ec2.internal
2026-09-16T19:41:22.963951515Z Collecting kubelet logs for node ip-10-0-137-37.ec2.internal
2026-09-16T19:41:22.965591021Z Collecting kubelet logs for node ip-10-0-139-120.ec2.internal
2026-09-16T19:41:22.974093767Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T19:41:23.352219205Z Gathering data for ns/openshift-apiserver...
2026-09-16T19:41:23.453767731Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-t4xfr namespace
2026-09-16T19:41:23.459082850Z INFO: Node performance data collection complete.
2026-09-16T19:41:23.798116947Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T19:41:24.066954512Z Gathering data for ns/openshift-controller-manager...
2026-09-16T19:41:24.336579993Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T19:41:24.980844465Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T19:41:25.274792854Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T19:41:25.582353195Z Gathering data for ns/openshift-service-ca...
2026-09-16T19:41:25.668530108Z [disk usage checker] Volume usage percentage: current = 34 ; allowed = 70
2026-09-16T19:41:26.006909215Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T19:41:26.911326210Z Wrote inspect data to must-gather.
2026-09-16T19:41:26.911567554Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:41:26.911567554Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T19:41:27.047586620Z error: the server doesn't have a resource type "clusters"
2026-09-16T19:41:27.202648974Z Caches written to disk