2026-05-29T20:38:26.139390510Z [disk usage checker] Started
2026-05-29T20:38:26.143342501Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T20:38:26.338487274Z Gathering data for ns/openshift-cluster-version...
2026-05-29T20:38:26.469652754Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T20:38:26.562927033Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T20:38:26.640585627Z Gathering data for ns/default...
2026-05-29T20:38:26.940459857Z Gathering data for ns/openshift...
2026-05-29T20:38:27.323587534Z Gathering data for ns/kube-system...
2026-05-29T20:38:28.038540419Z Gathering data for ns/openshift-etcd...
2026-05-29T20:38:28.325348395Z Waiting on subprocesses to finish execution.
2026-05-29T20:38:28.331026015Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-29T20:38:28.331358802Z Wrote inspect data to must-gather.
2026-05-29T20:38:28.331718361Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T20:38:28.331718361Z     namespaces "assisted-installer" not found
2026-05-29T20:38:28.339416849Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-29T20:38:28.344547128Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-29T20:38:28.366024975Z INFO: Collecting host service logs for crio
2026-05-29T20:38:28.366361788Z INFO: Collecting host service logs for kubelet
2026-05-29T20:38:28.366724846Z INFO: Collecting host service logs for rpm-ostreed
2026-05-29T20:38:28.367078555Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-29T20:38:28.367504297Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-29T20:38:28.367887639Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-29T20:38:28.368227970Z INFO: Collecting host service logs for NetworkManager
2026-05-29T20:38:28.368543687Z INFO: Collecting host service logs for openvswitch
2026-05-29T20:38:28.368898325Z INFO: Collecting host service logs for ovs-configuration
2026-05-29T20:38:28.369266415Z INFO: Collecting host service logs for ovsdb-server
2026-05-29T20:38:28.369601363Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-29T20:38:28.370115229Z INFO: Waiting for worker host service log collection to complete ...
2026-05-29T20:38:28.382537895Z INFO: Gathering HAProxy config files
2026-05-29T20:38:28.411556360Z INFO: Waiting for node performance related collection to complete ...
2026-05-29T20:38:28.695577430Z No resources found in openshift-etcd namespace.
2026-05-29T20:38:28.708166798Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-29T20:38:29.076445304Z INFO: "metallb-operator" not detected. Skipping.
2026-05-29T20:38:29.128967610Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-29T20:38:29.129052278Z INFO: on-disk MachineConfig config collection complete.
2026-05-29T20:38:29.262926424Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-29T20:38:29.362314708Z No resources found
2026-05-29T20:38:29.363204905Z error: the server doesn't have a resource type "performanceprofile"
2026-05-29T20:38:29.370310270Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-29T20:38:29.442154069Z Wrote inspect data to must-gather.
2026-05-29T20:38:29.451553835Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-6fs4g
2026-05-29T20:38:29.451611206Z insights-runtime-extractor-2t669
2026-05-29T20:38:29.451650849Z insights-runtime-extractor-jgzgd
2026-05-29T20:38:29.451676596Z insights-runtime-extractor-sv428
2026-05-29T20:38:29.739154612Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-29T20:38:29.855356941Z No resources found
2026-05-29T20:38:30.090552509Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.121715501Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-29T20:38:30.124307114Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-29T20:38:30.133798548Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-29T20:38:30.133798548Z See 'oc rsync -h' for help and examples
2026-05-29T20:38:30.135371535Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.149335240Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.190549464Z INFO: Worker host service log collection to complete.
2026-05-29T20:38:30.194075449Z Gathering data for ns/openshift-monitoring...
2026-05-29T20:38:30.221937051Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T20:38:30.237854270Z ERROR: No running kube-apiserver pods found
2026-05-29T20:38:30.339995382Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-29T20:38:30.380233511Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.497343538Z error: the server doesn't have a resource type "machineconfigs"
2026-05-29T20:38:30.531392058Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.538521919Z INFO: 'previous-logs' folder not found on ip-10-0-133-220.ec2.internal, skipping...
2026-05-29T20:38:30.550190199Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.631034215Z tar: Removing leading `/' from member names
2026-05-29T20:38:30.638573336Z INFO: HAProxy config collection complete.
2026-05-29T20:38:30.696085388Z INFO: Getting rules from prometheus-k8s-0
2026-05-29T20:38:30.735476347Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.798646335Z INFO: 'previous-logs' folder not found on ip-10-0-142-29.ec2.internal, skipping...
2026-05-29T20:38:30.816847359Z Wrote inspect data to must-gather.
2026-05-29T20:38:30.933655945Z Wrote inspect data to must-gather.
2026-05-29T20:38:31.048726610Z INFO: 'previous-logs' folder not found on ip-10-0-142-75.ec2.internal, skipping...
2026-05-29T20:38:31.048854644Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-29T20:38:31.048886610Z INFO: Machine Config Daemon termination log collection complete.
2026-05-29T20:38:31.070168627Z Wrote inspect data to must-gather.
2026-05-29T20:38:31.083313481Z INFO: Getting status/config from prometheus-k8s-0
2026-05-29T20:38:31.097614240Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-29T20:38:31.149592629Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T20:38:31.283742034Z Wrote inspect data to must-gather.
2026-05-29T20:38:31.287165991Z Wrote inspect data to must-gather.
2026-05-29T20:38:31.476705495Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-29T20:38:31.481328320Z Wrote inspect data to must-gather.
2026-05-29T20:38:31.542306546Z INFO: INTERCONNECT MODE
2026-05-29T20:38:31.542337157Z INFO: Gathering ovn-kubernetes DBs
2026-05-29T20:38:31.695937150Z INFO: Gathering OVN_Northbound from ovnkube-node-4tfqw...
2026-05-29T20:38:31.696168049Z INFO: Gathering OVN_Northbound from ovnkube-node-f6gs2...
2026-05-29T20:38:31.696409763Z INFO: Gathering OVN_Northbound from ovnkube-node-gb9vl...
2026-05-29T20:38:31.696706398Z INFO: Gathering OVN_Southbound from ovnkube-node-4tfqw...
2026-05-29T20:38:31.696954692Z INFO: Gathering OVN_Southbound from ovnkube-node-f6gs2...
2026-05-29T20:38:31.697277987Z INFO: Gathering OVN_Southbound from ovnkube-node-gb9vl...
2026-05-29T20:38:31.697959412Z Wrote inspect data to must-gather.
2026-05-29T20:38:31.709130777Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-29T20:38:31.772467911Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T20:38:31.996269044Z tar: Removing leading `/' from member names
2026-05-29T20:38:32.030989876Z tar: Removing leading `/' from member names
2026-05-29T20:38:32.047857894Z tar: Removing leading `/' from member names
2026-05-29T20:38:32.048780472Z tar: Removing leading `/' from member names
2026-05-29T20:38:32.051704816Z tar: Removing leading `/' from member names
2026-05-29T20:38:32.055317288Z tar: Removing leading `/' from member names
2026-05-29T20:38:32.060420281Z Wrote inspect data to must-gather.
2026-05-29T20:38:32.102848338Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-29T20:38:32.282059011Z Wrote inspect data to must-gather.
2026-05-29T20:38:32.357237287Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-29T20:38:32.488951213Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T20:38:32.489738279Z INFO: Waiting for network log collection to complete ...
2026-05-29T20:38:32.489781416Z INFO: Waiting for ovnk database copies to complete ...
2026-05-29T20:38:32.490179618Z INFO: Copying ovnk databases complete.
2026-05-29T20:38:32.491804524Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-05-29T20:38:32.493845067Z ovnk_database_store/
2026-05-29T20:38:32.493845067Z ovnk_database_store/ovnkube-node-gb9vl_nbdb
2026-05-29T20:38:32.502128723Z ovnk_database_store/ovnkube-node-f6gs2_nbdb
2026-05-29T20:38:32.507784655Z ovnk_database_store/ovnkube-node-f6gs2_sbdb
2026-05-29T20:38:32.524399866Z ovnk_database_store/ovnkube-node-gb9vl_sbdb
2026-05-29T20:38:32.538690244Z ovnk_database_store/ovnkube-node-4tfqw_sbdb
2026-05-29T20:38:32.553311193Z ovnk_database_store/ovnkube-node-4tfqw_nbdb
2026-05-29T20:38:32.561179184Z INFO: Network log collection complete.
2026-05-29T20:38:32.576113916Z Wrote inspect data to must-gather.
2026-05-29T20:38:32.645876597Z Gathering data for ns/openshift-network-console...
2026-05-29T20:38:32.725925984Z INFO: Getting status from alertmanager-main-0
2026-05-29T20:38:32.841940325Z Wrote inspect data to must-gather.
2026-05-29T20:38:33.073402533Z Gathering data for ns/openshift-console-operator...
2026-05-29T20:38:33.089014710Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-29T20:38:33.091843326Z INFO: Fallback to identify the container image from release info
2026-05-29T20:38:33.454680302Z Gathering data for ns/openshift-console...
2026-05-29T20:38:33.834126477Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-29T20:38:33.863295503Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-05-29T20:38:33.988669310Z daemonset.apps/perf-node-gather-daemonset created
2026-05-29T20:38:34.106063880Z Waiting for performance profile collector pods to become ready: 1
2026-05-29T20:38:34.179218907Z Gathering data for ns/openshift-dns-operator...
2026-05-29T20:38:34.444462742Z Gathering data for ns/openshift-dns...
2026-05-29T20:38:35.038890752Z Gathering data for ns/openshift-image-registry...
2026-05-29T20:38:35.223870690Z Waiting for performance profile collector pods to become ready: 2
2026-05-29T20:38:35.475183265Z Gathering data for ns/openshift-ingress-operator...
2026-05-29T20:38:35.774286625Z Gathering data for ns/openshift-ingress...
2026-05-29T20:38:36.073146842Z Gathering data for ns/openshift-ingress-canary...
2026-05-29T20:38:36.158877476Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T20:38:36.331904629Z Waiting for performance profile collector pods to become ready: 3
2026-05-29T20:38:36.465430901Z Gathering data for ns/openshift-insights...
2026-05-29T20:38:37.438435178Z Waiting for performance profile collector pods to become ready: 4
2026-05-29T20:38:38.197038428Z Gathering data for ns/openshift-config...
2026-05-29T20:38:38.440582979Z Gathering data for ns/openshift-config-managed...
2026-05-29T20:38:38.567022309Z Waiting for performance profile collector pods to become ready: 5
2026-05-29T20:38:38.886436606Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-29T20:38:39.142530548Z Gathering data for ns/openshift-kube-apiserver...
2026-05-29T20:38:39.392126026Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-29T20:38:39.661925740Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-29T20:38:39.689834447Z Waiting for performance profile collector pods to become ready: 6
2026-05-29T20:38:39.963618281Z Gathering data for ns/openshift-kube-scheduler...
2026-05-29T20:38:40.215956828Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-29T20:38:40.491743783Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-29T20:38:40.810655171Z Waiting for performance profile collector pods to become ready: 7
2026-05-29T20:38:40.861230991Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-29T20:38:41.164174422Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T20:38:41.167478367Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-29T20:38:41.659062867Z Gathering data for ns/openshift-multus...
2026-05-29T20:38:41.916753636Z Waiting for performance profile collector pods to become ready: 8
2026-05-29T20:38:42.945297913Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-29T20:38:43.020654898Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-29T20:38:43.245924850Z Collecting performance related data for node ip-10-0-142-75.ec2.internal
2026-05-29T20:38:43.252744734Z Collecting performance related data for node ip-10-0-133-220.ec2.internal
2026-05-29T20:38:43.259782409Z Collecting performance related data for node ip-10-0-142-29.ec2.internal
2026-05-29T20:38:44.586922272Z Gathering data for ns/openshift-host-network...
2026-05-29T20:38:44.909668200Z Gathering data for ns/openshift-network-diagnostics...
2026-05-29T20:38:45.411201445Z Gathering data for ns/openshift-network-node-identity...
2026-05-29T20:38:45.854023353Z Gathering data for ns/openshift-network-operator...
2026-05-29T20:38:46.169361843Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T20:38:46.259375264Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-29T20:38:46.520495603Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-29T20:38:46.944213236Z Gathering data for ns/openshift-apiserver-operator...
2026-05-29T20:38:47.241592754Z Gathering data for ns/openshift-apiserver...
2026-05-29T20:38:47.601918216Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-29T20:38:47.864801298Z Gathering data for ns/openshift-controller-manager...
2026-05-29T20:38:48.143037284Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-29T20:38:48.789154190Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-29T20:38:49.102815359Z Gathering data for ns/openshift-service-ca-operator...
2026-05-29T20:38:49.421761117Z Gathering data for ns/openshift-service-ca...
2026-05-29T20:38:49.765156729Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-29T20:38:50.629459356Z Wrote inspect data to must-gather.
2026-05-29T20:38:50.629587468Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T20:38:50.629587468Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-29T20:38:51.129494621Z Collecting kubelet logs for node ip-10-0-142-75.ec2.internal
2026-05-29T20:38:51.131267117Z Collecting kubelet logs for node ip-10-0-133-220.ec2.internal
2026-05-29T20:38:51.132892865Z Collecting kubelet logs for node ip-10-0-142-29.ec2.internal
2026-05-29T20:38:51.174094001Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T20:38:51.417996252Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-29T20:38:51.423331189Z INFO: Node performance data collection complete.
2026-05-29T20:38:51.548498063Z error: the server doesn't have a resource type "clusters"
2026-05-29T20:38:51.699675624Z Caches written to disk