2026-04-22T20:59:57.073600692Z [disk usage checker] Started
2026-04-22T20:59:57.077267505Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T20:59:57.366533238Z Gathering data for ns/openshift-cluster-version...
2026-04-22T20:59:57.497480972Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T20:59:57.571953113Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T20:59:57.651239674Z Gathering data for ns/default...
2026-04-22T20:59:57.919894704Z Gathering data for ns/openshift...
2026-04-22T20:59:58.351835726Z Gathering data for ns/kube-system...
2026-04-22T20:59:59.018442925Z Gathering data for ns/openshift-etcd...
2026-04-22T20:59:59.343312794Z Wrote inspect data to must-gather.
2026-04-22T20:59:59.343344252Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T20:59:59.343344252Z     namespaces "assisted-installer" not found
2026-04-22T20:59:59.634510649Z Waiting on subprocesses to finish execution.
2026-04-22T20:59:59.638568094Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T20:59:59.647615284Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T20:59:59.655157891Z INFO: Gathering HAProxy config files
2026-04-22T20:59:59.660245729Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T20:59:59.669601301Z INFO: Collecting host service logs for crio
2026-04-22T20:59:59.669915634Z INFO: Collecting host service logs for kubelet
2026-04-22T20:59:59.670214231Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T20:59:59.670575919Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T20:59:59.670937377Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T20:59:59.671367331Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T20:59:59.671787710Z INFO: Collecting host service logs for NetworkManager
2026-04-22T20:59:59.672142258Z INFO: Collecting host service logs for openvswitch
2026-04-22T20:59:59.672498072Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T20:59:59.672835251Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T20:59:59.673208076Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T20:59:59.673731264Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T20:59:59.691994325Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T21:00:00.119099309Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T21:00:00.396825424Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T21:00:00.631974252Z No resources found in openshift-etcd namespace.
2026-04-22T21:00:00.785279857Z INFO: Worker host service log collection to complete.
2026-04-22T21:00:00.852614041Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T21:00:00.852698282Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T21:00:00.967803780Z No resources found
2026-04-22T21:00:01.023309283Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T21:00:01.112008674Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T21:00:01.144153784Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T21:00:01.172198261Z Wrote inspect data to must-gather.
2026-04-22T21:00:01.261501731Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-br2nv
2026-04-22T21:00:01.261552768Z insights-runtime-extractor-8wd72
2026-04-22T21:00:01.261572962Z insights-runtime-extractor-b8fc9
2026-04-22T21:00:01.261590486Z insights-runtime-extractor-rjvgq
2026-04-22T21:00:01.263194879Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T21:00:01.265678831Z Wrote inspect data to must-gather.
2026-04-22T21:00:01.401221850Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T21:00:01.423210508Z Wrote inspect data to must-gather.
2026-04-22T21:00:01.630251746Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T21:00:01.636644974Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T21:00:01.759677254Z Gathering data for ns/openshift-monitoring...
2026-04-22T21:00:01.807827094Z Wrote inspect data to must-gather.
2026-04-22T21:00:01.808340838Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T21:00:01.808340838Z See 'oc rsync -h' for help and examples
2026-04-22T21:00:01.812263761Z No resources found
2026-04-22T21:00:01.871463517Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T21:00:01.967361249Z ERROR: No running kube-apiserver pods found
2026-04-22T21:00:01.971601918Z INFO: 'previous-logs' folder not found on ip-10-0-128-61.ec2.internal, skipping...
2026-04-22T21:00:02.014633438Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.051106250Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.052654425Z tar: Removing leading `/' from member names
2026-04-22T21:00:02.060616762Z INFO: HAProxy config collection complete.
2026-04-22T21:00:02.079925459Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.083648447Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:02.114340757Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T21:00:02.261921878Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T21:00:02.314841299Z INFO: 'previous-logs' folder not found on ip-10-0-135-221.ec2.internal, skipping...
2026-04-22T21:00:02.339891683Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.513954329Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.579989833Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T21:00:02.600906744Z INFO: 'previous-logs' folder not found on ip-10-0-141-46.ec2.internal, skipping...
2026-04-22T21:00:02.600943742Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T21:00:02.600970861Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T21:00:02.613848333Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.764060182Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.847941520Z Wrote inspect data to must-gather.
2026-04-22T21:00:02.933452484Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T21:00:02.998168825Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T21:00:03.028432305Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T21:00:03.170314248Z Wrote inspect data to must-gather.
2026-04-22T21:00:03.174593788Z INFO: INTERCONNECT MODE
2026-04-22T21:00:03.174620204Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T21:00:03.227220270Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T21:00:03.346097859Z INFO: Gathering OVN_Northbound from ovnkube-node-cw48v...
2026-04-22T21:00:03.346379293Z INFO: Gathering OVN_Northbound from ovnkube-node-mpl4p...
2026-04-22T21:00:03.346654690Z INFO: Gathering OVN_Northbound from ovnkube-node-ws7ww...
2026-04-22T21:00:03.347085822Z INFO: Gathering OVN_Southbound from ovnkube-node-cw48v...
2026-04-22T21:00:03.347497399Z INFO: Gathering OVN_Southbound from ovnkube-node-mpl4p...
2026-04-22T21:00:03.347821827Z INFO: Gathering OVN_Southbound from ovnkube-node-ws7ww...
2026-04-22T21:00:03.438176166Z Wrote inspect data to must-gather.
2026-04-22T21:00:03.525382539Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T21:00:03.633080762Z tar: Removing leading `/' from member names
2026-04-22T21:00:03.642188979Z tar: Removing leading `/' from member names
2026-04-22T21:00:03.642647187Z tar: Removing leading `/' from member names
2026-04-22T21:00:03.653253124Z tar: Removing leading `/' from member names
2026-04-22T21:00:03.699569425Z tar: Removing leading `/' from member names
2026-04-22T21:00:03.716784507Z tar: Removing leading `/' from member names
2026-04-22T21:00:03.780061428Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T21:00:03.828214331Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T21:00:03.898656761Z Wrote inspect data to must-gather.
2026-04-22T21:00:03.994193354Z Gathering data for ns/openshift-network-console...
2026-04-22T21:00:04.189522876Z INFO: Waiting for network log collection to complete ...
2026-04-22T21:00:04.189560113Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T21:00:04.190028320Z INFO: Copying ovnk databases complete.
2026-04-22T21:00:04.191841760Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-04-22T21:00:04.194725879Z ovnk_database_store/
2026-04-22T21:00:04.194773260Z ovnk_database_store/ovnkube-node-mpl4p_nbdb
2026-04-22T21:00:04.209362758Z ovnk_database_store/ovnkube-node-cw48v_sbdb
2026-04-22T21:00:04.234697595Z Wrote inspect data to must-gather.
2026-04-22T21:00:04.234802707Z ovnk_database_store/ovnkube-node-cw48v_nbdb
2026-04-22T21:00:04.247286981Z ovnk_database_store/ovnkube-node-mpl4p_sbdb
2026-04-22T21:00:04.248906994Z INFO: Getting status from alertmanager-main-0
2026-04-22T21:00:04.271892909Z ovnk_database_store/ovnkube-node-ws7ww_nbdb
2026-04-22T21:00:04.285456828Z ovnk_database_store/ovnkube-node-ws7ww_sbdb
2026-04-22T21:00:04.318595572Z INFO: Network log collection complete.
2026-04-22T21:00:04.430949394Z Gathering data for ns/openshift-console-operator...
2026-04-22T21:00:04.548202057Z Wrote inspect data to must-gather.
2026-04-22T21:00:04.779439209Z Gathering data for ns/openshift-console...
2026-04-22T21:00:04.841475469Z Wrote inspect data to must-gather.
2026-04-22T21:00:05.131486540Z Wrote inspect data to must-gather.
2026-04-22T21:00:05.207217218Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T21:00:05.395930810Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T21:00:05.399912407Z INFO: Fallback to identify the container image from release info
2026-04-22T21:00:05.550215216Z Gathering data for ns/openshift-dns-operator...
2026-04-22T21:00:05.792741110Z Gathering data for ns/openshift-dns...
2026-04-22T21:00:06.194738342Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T21:00:06.331262081Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T21:00:06.394556151Z Gathering data for ns/openshift-image-registry...
2026-04-22T21:00:06.475851851Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T21:00:06.919007516Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T21:00:07.092460209Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:12.097602022Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:17.102416064Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:17.562941565Z E0422 21:00:17.562899    1534 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://172.31.0.1:443/api?timeout=32s\": net/http: TLS handshake timeout"
2026-04-22T21:00:17.592496918Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T21:00:17.822681316Z Collecting performance related data for node ip-10-0-135-221.ec2.internal
2026-04-22T21:00:17.829842417Z Collecting performance related data for node ip-10-0-141-46.ec2.internal
2026-04-22T21:00:17.837363040Z Collecting performance related data for node ip-10-0-128-61.ec2.internal
2026-04-22T21:00:22.107138783Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:27.111888905Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:28.208799307Z E0422 21:00:28.208741    1648 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://172.31.0.1:443/api?timeout=32s\": net/http: TLS handshake timeout"
2026-04-22T21:00:28.433389281Z E0422 21:00:28.433348    1690 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://172.31.0.1:443/api?timeout=32s\": net/http: TLS handshake timeout"
2026-04-22T21:00:32.116628939Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:37.121487579Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:38.214398207Z E0422 21:00:38.214341    1648 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://172.31.0.1:443/api?timeout=32s\": net/http: TLS handshake timeout"
2026-04-22T21:00:42.126160282Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:47.130920114Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:48.767926374Z E0422 21:00:48.767888    1759 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://172.31.0.1:443/api?timeout=32s\": net/http: TLS handshake timeout"
2026-04-22T21:00:48.791134416Z E0422 21:00:48.791092    1770 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://172.31.0.1:443/api?timeout=32s\": net/http: TLS handshake timeout"
2026-04-22T21:00:52.136323370Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:53.224244994Z Gathering data for ns/openshift-ingress...
2026-04-22T21:00:53.531598713Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T21:00:53.871957802Z Gathering data for ns/openshift-insights...
2026-04-22T21:00:55.706411466Z Collecting kubelet logs for node ip-10-0-135-221.ec2.internal
2026-04-22T21:00:55.708272494Z Collecting kubelet logs for node ip-10-0-141-46.ec2.internal
2026-04-22T21:00:55.710480475Z Collecting kubelet logs for node ip-10-0-128-61.ec2.internal
2026-04-22T21:00:55.937025169Z Gathering data for ns/kserve...
2026-04-22T21:00:56.152970348Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T21:00:56.159453066Z INFO: Node performance data collection complete.
2026-04-22T21:00:57.141102030Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:00:57.956583544Z Gathering data for ns/openshift-config...
2026-04-22T21:00:58.195078023Z Gathering data for ns/openshift-config-managed...
2026-04-22T21:00:58.580591740Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T21:00:58.805736189Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T21:00:59.031634849Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T21:00:59.288045116Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T21:00:59.582567066Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T21:00:59.806015010Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T21:01:00.057955947Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T21:01:00.336634757Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T21:01:00.597186167Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T21:01:01.048962087Z Gathering data for ns/openshift-multus...
2026-04-22T21:01:02.147085596Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:01:02.232919390Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T21:01:03.611308374Z Gathering data for ns/openshift-host-network...
2026-04-22T21:01:03.865809349Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T21:01:04.326610818Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T21:01:04.672038756Z Gathering data for ns/openshift-network-operator...
2026-04-22T21:01:04.997847169Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T21:01:05.229128759Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T21:01:05.626639591Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T21:01:05.888942163Z Gathering data for ns/openshift-apiserver...
2026-04-22T21:01:06.222782115Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T21:01:06.445997794Z Gathering data for ns/openshift-controller-manager...
2026-04-22T21:01:06.676974929Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T21:01:07.153259161Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-22T21:01:07.286199929Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T21:01:07.582825897Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T21:01:07.918385596Z Gathering data for ns/openshift-service-ca...
2026-04-22T21:01:08.239465053Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T21:01:09.078538716Z Wrote inspect data to must-gather.
2026-04-22T21:01:09.078584699Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T21:01:09.078584699Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T21:01:09.214904086Z error: the server doesn't have a resource type "clusters"
2026-04-22T21:01:09.348270134Z Caches written to disk