2026-07-27T17:56:00.356185763Z [disk usage checker] Started
2026-07-27T17:56:00.359421939Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T17:56:00.566212722Z Gathering data for ns/openshift-cluster-version...
2026-07-27T17:56:00.718385671Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-27T17:56:00.828940645Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T17:56:00.913284884Z Gathering data for ns/default...
2026-07-27T17:56:01.236586667Z Gathering data for ns/openshift...
2026-07-27T17:56:01.651227212Z Gathering data for ns/kube-system...
2026-07-27T17:56:02.395953003Z Gathering data for ns/openshift-etcd...
2026-07-27T17:56:02.714672062Z Wrote inspect data to must-gather.
2026-07-27T17:56:02.714700253Z error: inspection completed with the errors occurred while gathering data:
2026-07-27T17:56:02.714700253Z     namespaces "assisted-installer" not found
2026-07-27T17:56:02.964605131Z Waiting on subprocesses to finish execution.
2026-07-27T17:56:02.968867914Z INFO: Gathering HAProxy config files
2026-07-27T17:56:02.970670551Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-27T17:56:02.984258674Z INFO: Collecting host service logs for crio
2026-07-27T17:56:02.984602599Z INFO: Collecting host service logs for kubelet
2026-07-27T17:56:02.984915880Z INFO: Collecting host service logs for rpm-ostreed
2026-07-27T17:56:02.985241965Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-27T17:56:02.985559116Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-27T17:56:02.985855768Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-27T17:56:02.986197707Z INFO: Collecting host service logs for NetworkManager
2026-07-27T17:56:02.986493947Z INFO: Collecting host service logs for openvswitch
2026-07-27T17:56:02.986775946Z INFO: Collecting host service logs for ovs-configuration
2026-07-27T17:56:02.987073187Z INFO: Collecting host service logs for ovsdb-server
2026-07-27T17:56:02.987404280Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-27T17:56:02.987801966Z INFO: Waiting for worker host service log collection to complete ...
2026-07-27T17:56:02.987941030Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-27T17:56:03.012187937Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-27T17:56:03.017855050Z Executing Istio gather script
2026-07-27T17:56:03.119876226Z INFO: Waiting for node performance related collection to complete ...
2026-07-27T17:56:03.730368858Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-qrbk4
2026-07-27T17:56:03.730368858Z insights-runtime-extractor-6jsxl
2026-07-27T17:56:03.730368858Z insights-runtime-extractor-bkc9q
2026-07-27T17:56:03.730368858Z insights-runtime-extractor-dl489
2026-07-27T17:56:03.975997958Z No resources found
2026-07-27T17:56:03.987775967Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-27T17:56:03.987866211Z INFO: on-disk MachineConfig config collection complete.
2026-07-27T17:56:04.072896064Z Wrote inspect data to must-gather.
2026-07-27T17:56:04.134836689Z INFO: "metallb-operator" not detected. Skipping.
2026-07-27T17:56:04.163466947Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-27T17:56:04.226635204Z No resources found in openshift-etcd namespace.
2026-07-27T17:56:04.272344787Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-27T17:56:04.315454677Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-27T17:56:04.317097225Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-27T17:56:04.525186782Z INFO: Worker host service log collection to complete.
2026-07-27T17:56:04.525961140Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-27T17:56:04.561066227Z error: the server doesn't have a resource type "performanceprofile"
2026-07-27T17:56:04.671493110Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-27T17:56:04.671493110Z See 'oc rsync -h' for help and examples
2026-07-27T17:56:04.958958541Z Wrote inspect data to must-gather.
2026-07-27T17:56:04.988386312Z ERROR: No running kube-apiserver pods found
2026-07-27T17:56:05.002496865Z tar: Removing leading `/' from member names
2026-07-27T17:56:05.008220270Z INFO: HAProxy config collection complete.
2026-07-27T17:56:05.013680779Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-27T17:56:05.192255972Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-27T17:56:05.209034581Z Wrote inspect data to must-gather.
2026-07-27T17:56:05.237430901Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T17:56:05.264738788Z No resources found
2026-07-27T17:56:05.414682167Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T17:56:05.438679228Z Wrote inspect data to must-gather.
2026-07-27T17:56:05.505185632Z INFO: 'previous-logs' folder not found on ip-10-0-135-237.ec2.internal, skipping...
2026-07-27T17:56:05.528346157Z Wrote inspect data to must-gather.
2026-07-27T17:56:05.572098584Z Wrote inspect data to must-gather.
2026-07-27T17:56:05.588641288Z Wrote inspect data to must-gather.
2026-07-27T17:56:05.690096536Z error: the server doesn't have a resource type "machineconfigs"
2026-07-27T17:56:05.697102750Z Gathering data for ns/openshift-monitoring...
2026-07-27T17:56:05.845222200Z INFO: Getting rules from prometheus-k8s-0
2026-07-27T17:56:05.865201934Z INFO: 'previous-logs' folder not found on ip-10-0-139-217.ec2.internal, skipping...
2026-07-27T17:56:05.891817103Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-27T17:56:05.944863851Z Wrote inspect data to must-gather.
2026-07-27T17:56:06.059149294Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-27T17:56:06.143462847Z Wrote inspect data to must-gather.
2026-07-27T17:56:06.178662994Z INFO: 'previous-logs' folder not found on ip-10-0-140-197.ec2.internal, skipping...
2026-07-27T17:56:06.178744680Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-27T17:56:06.178744680Z INFO: Machine Config Daemon termination log collection complete.
2026-07-27T17:56:06.223893997Z Wrote inspect data to must-gather.
2026-07-27T17:56:06.284613031Z INFO: Getting status/config from prometheus-k8s-0
2026-07-27T17:56:06.303082117Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-27T17:56:06.493434384Z Wrote inspect data to must-gather.
2026-07-27T17:56:06.515168981Z Wrote inspect data to must-gather.
2026-07-27T17:56:06.694113865Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-27T17:56:06.725798671Z Wrote inspect data to must-gather.
2026-07-27T17:56:06.787274342Z INFO: INTERCONNECT MODE
2026-07-27T17:56:06.787329125Z INFO: Gathering ovn-kubernetes DBs
2026-07-27T17:56:06.858017268Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-27T17:56:06.923672105Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-27T17:56:06.948257680Z Wrote inspect data to must-gather.
2026-07-27T17:56:06.957756497Z INFO: Gathering OVN_Northbound from ovnkube-node-79zhz...
2026-07-27T17:56:06.958005197Z INFO: Gathering OVN_Northbound from ovnkube-node-7z9fr...
2026-07-27T17:56:06.958286727Z INFO: Gathering OVN_Northbound from ovnkube-node-c2rk9...
2026-07-27T17:56:06.958551653Z INFO: Gathering OVN_Southbound from ovnkube-node-79zhz...
2026-07-27T17:56:06.958818432Z INFO: Gathering OVN_Southbound from ovnkube-node-7z9fr...
2026-07-27T17:56:06.959080584Z INFO: Gathering OVN_Southbound from ovnkube-node-c2rk9...
2026-07-27T17:56:07.135836922Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:07.279392660Z tar: Removing leading `/' from member names
2026-07-27T17:56:07.293875032Z tar: Removing leading `/' from member names
2026-07-27T17:56:07.294689888Z tar: Removing leading `/' from member names
2026-07-27T17:56:07.300808635Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-27T17:56:07.326149789Z tar: Removing leading `/' from member names
2026-07-27T17:56:07.359251538Z Wrote inspect data to must-gather.
2026-07-27T17:56:07.378242431Z tar: Removing leading `/' from member names
2026-07-27T17:56:07.378339234Z tar: Removing leading `/' from member names
2026-07-27T17:56:07.424449234Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-27T17:56:07.450334452Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:07.453638037Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-27T17:56:07.585584708Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-27T17:56:07.608700697Z Wrote inspect data to must-gather.
2026-07-27T17:56:07.699575297Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:07.862625456Z Wrote inspect data to must-gather.
2026-07-27T17:56:07.890239647Z INFO: Waiting for network log collection to complete ...
2026-07-27T17:56:07.890271265Z INFO: Waiting for ovnk database copies to complete ...
2026-07-27T17:56:07.890746378Z INFO: Copying ovnk databases complete.
2026-07-27T17:56:07.894048037Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-27T17:56:07.899477338Z ovnk_database_store/
2026-07-27T17:56:07.899530748Z ovnk_database_store/ovnkube-node-79zhz_sbdb
2026-07-27T17:56:07.919814395Z ovnk_database_store/ovnkube-node-c2rk9_sbdb
2026-07-27T17:56:07.936094883Z ovnk_database_store/ovnkube-node-c2rk9_nbdb
2026-07-27T17:56:07.939775370Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:07.943727266Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-27T17:56:07.944027679Z ovnk_database_store/ovnkube-node-79zhz_nbdb
2026-07-27T17:56:07.951217100Z ovnk_database_store/ovnkube-node-7z9fr_sbdb
2026-07-27T17:56:07.968722017Z ovnk_database_store/ovnkube-node-7z9fr_nbdb
2026-07-27T17:56:07.977439170Z INFO: Network log collection complete.
2026-07-27T17:56:08.042751589Z INFO: Getting status from alertmanager-main-0
2026-07-27T17:56:08.093465723Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T17:56:08.177855007Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:08.180482370Z Wrote inspect data to must-gather.
2026-07-27T17:56:08.275680313Z Gathering data for ns/openshift-network-console...
2026-07-27T17:56:08.384946898Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:08.389626435Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-27T17:56:08.458259602Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-27T17:56:08.460384499Z INFO: Fallback to identify the container image from release info
2026-07-27T17:56:08.620668690Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:08.723003958Z Gathering data for ns/openshift-console-operator...
2026-07-27T17:56:08.814714681Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:08.817697642Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-27T17:56:08.958074640Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:09.156466342Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:56:09.165084680Z Gathering data for ns/openshift-console...
2026-07-27T17:56:09.204296982Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-27T17:56:09.341275817Z daemonset.apps/perf-node-gather-daemonset created
2026-07-27T17:56:09.458336114Z Waiting for performance profile collector pods to become ready: 1
2026-07-27T17:56:09.742902230Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-27T17:56:09.891965565Z Done executing Istio gather script
2026-07-27T17:56:10.230554259Z Gathering data for ns/openshift-dns-operator...
2026-07-27T17:56:10.426374664Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T17:56:10.500385573Z Gathering data for ns/openshift-dns...
2026-07-27T17:56:10.569441565Z Waiting for performance profile collector pods to become ready: 2
2026-07-27T17:56:11.067173394Z Gathering data for ns/openshift-image-registry...
2026-07-27T17:56:11.497567880Z Gathering data for ns/openshift-ingress-operator...
2026-07-27T17:56:11.680081599Z Waiting for performance profile collector pods to become ready: 3
2026-07-27T17:56:11.811949588Z Gathering data for ns/openshift-ingress...
2026-07-27T17:56:12.121313693Z Gathering data for ns/openshift-ingress-canary...
2026-07-27T17:56:12.497274082Z Gathering data for ns/openshift-insights...
2026-07-27T17:56:12.794707798Z Waiting for performance profile collector pods to become ready: 4
2026-07-27T17:56:13.904563979Z Waiting for performance profile collector pods to become ready: 5
2026-07-27T17:56:14.364220102Z Gathering data for ns/openshift-jobset-operator...
2026-07-27T17:56:15.023744735Z Waiting for performance profile collector pods to become ready: 6
2026-07-27T17:56:15.254273649Z Gathering data for ns/openshift-config...
2026-07-27T17:56:15.432606543Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T17:56:15.544119848Z Gathering data for ns/openshift-config-managed...
2026-07-27T17:56:15.968868183Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-27T17:56:16.126472505Z Waiting for performance profile collector pods to become ready: 7
2026-07-27T17:56:16.286135718Z Gathering data for ns/openshift-kube-apiserver...
2026-07-27T17:56:16.543924857Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-27T17:56:16.884514969Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-27T17:56:17.182895483Z Gathering data for ns/openshift-kube-scheduler...
2026-07-27T17:56:17.260512666Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-27T17:56:17.511065257Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-27T17:56:17.556112818Z Collecting performance related data for node ip-10-0-140-197.ec2.internal
2026-07-27T17:56:17.569234859Z Collecting performance related data for node ip-10-0-135-237.ec2.internal
2026-07-27T17:56:17.586875952Z Collecting performance related data for node ip-10-0-139-217.ec2.internal
2026-07-27T17:56:17.855196724Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-27T17:56:18.280013310Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-27T17:56:18.683462532Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-27T17:56:19.245062308Z Gathering data for ns/openshift-multus...
2026-07-27T17:56:20.439563717Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T17:56:20.747488016Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-27T17:56:22.047023795Z Gathering data for ns/openshift-host-network...
2026-07-27T17:56:22.363298530Z Gathering data for ns/openshift-network-diagnostics...
2026-07-27T17:56:22.893794596Z Gathering data for ns/openshift-network-node-identity...
2026-07-27T17:56:23.311134810Z Gathering data for ns/openshift-network-operator...
2026-07-27T17:56:23.691765396Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-27T17:56:23.948329065Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-27T17:56:24.427673104Z Gathering data for ns/openshift-apiserver-operator...
2026-07-27T17:56:24.762422726Z Gathering data for ns/openshift-apiserver...
2026-07-27T17:56:25.165657600Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-27T17:56:25.445993539Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-27T17:56:25.473544591Z Gathering data for ns/openshift-controller-manager...
2026-07-27T17:56:25.764391545Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-27T17:56:26.039343113Z Collecting kubelet logs for node ip-10-0-140-197.ec2.internal
2026-07-27T17:56:26.041187959Z Collecting kubelet logs for node ip-10-0-135-237.ec2.internal
2026-07-27T17:56:26.043036375Z Collecting kubelet logs for node ip-10-0-139-217.ec2.internal
2026-07-27T17:56:26.386774219Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vwpgj namespace
2026-07-27T17:56:26.391787541Z INFO: Node performance data collection complete.
2026-07-27T17:56:26.484430049Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-27T17:56:26.826073020Z Gathering data for ns/openshift-service-ca-operator...
2026-07-27T17:56:27.149242672Z Gathering data for ns/openshift-service-ca...
2026-07-27T17:56:27.528881835Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-27T17:56:28.427814723Z Wrote inspect data to must-gather.
2026-07-27T17:56:28.427864249Z error: inspection completed with the errors occurred while gathering data:
2026-07-27T17:56:28.427864249Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-27T17:56:28.554738348Z error: the server doesn't have a resource type "clusters"
2026-07-27T17:56:28.710896872Z Caches written to disk