2026-04-22T15:16:30.484994527Z [disk usage checker] Started
2026-04-22T15:16:30.514653898Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-04-22T15:16:30.847025773Z Gathering data for ns/openshift-cluster-version...
2026-04-22T15:16:30.983186615Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T15:16:31.076444994Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T15:16:31.159302935Z Gathering data for ns/default...
2026-04-22T15:16:31.506194554Z Gathering data for ns/openshift...
2026-04-22T15:16:31.929032549Z Gathering data for ns/kube-system...
2026-04-22T15:16:32.755311792Z Gathering data for ns/openshift-etcd...
2026-04-22T15:16:33.093126055Z Wrote inspect data to must-gather.
2026-04-22T15:16:33.093152241Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T15:16:33.093152241Z     namespaces "assisted-installer" not found
2026-04-22T15:16:33.114898599Z Waiting on subprocesses to finish execution.
2026-04-22T15:16:33.122219366Z INFO: Gathering HAProxy config files
2026-04-22T15:16:33.125368284Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T15:16:33.129297570Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T15:16:33.132438915Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T15:16:33.143927905Z INFO: Collecting host service logs for crio
2026-04-22T15:16:33.144206219Z INFO: Collecting host service logs for kubelet
2026-04-22T15:16:33.144541908Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T15:16:33.144861778Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T15:16:33.145161148Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T15:16:33.145452343Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T15:16:33.145859005Z INFO: Collecting host service logs for NetworkManager
2026-04-22T15:16:33.146189800Z INFO: Collecting host service logs for openvswitch
2026-04-22T15:16:33.146482871Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T15:16:33.146805000Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T15:16:33.147094446Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T15:16:33.147450365Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T15:16:33.202361208Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T15:16:33.707434770Z No resources found in openshift-etcd namespace.
2026-04-22T15:16:33.809911083Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T15:16:33.941960385Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T15:16:33.990742574Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T15:16:33.994596955Z Wrote inspect data to must-gather.
2026-04-22T15:16:34.276176738Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T15:16:34.319031489Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T15:16:34.501927986Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T15:16:34.503655321Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T15:16:34.631317877Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T15:16:34.791956996Z ERROR: No running kube-apiserver pods found
2026-04-22T15:16:34.862569080Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T15:16:34.870193380Z Wrote inspect data to must-gather.
2026-04-22T15:16:34.878991174Z Gathering data for ns/openshift-monitoring...
2026-04-22T15:16:35.002834736Z Wrote inspect data to must-gather.
2026-04-22T15:16:35.015107476Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T15:16:35.065814047Z Wrote inspect data to must-gather.
2026-04-22T15:16:35.161568186Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-2zk62
2026-04-22T15:16:35.161616374Z insights-runtime-extractor-5d5wd
2026-04-22T15:16:35.161636250Z insights-runtime-extractor-c9nv9
2026-04-22T15:16:35.161655425Z insights-runtime-extractor-sgkpp
2026-04-22T15:16:35.215202922Z tar: Removing leading `/' from member names
2026-04-22T15:16:35.223342400Z INFO: HAProxy config collection complete.
2026-04-22T15:16:35.245558399Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T15:16:35.250304760Z No resources found
2026-04-22T15:16:35.443159479Z No resources found
2026-04-22T15:16:35.459924160Z INFO: Worker host service log collection to complete.
2026-04-22T15:16:35.476922796Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T15:16:35.541863377Z Wrote inspect data to must-gather.
2026-04-22T15:16:35.552139204Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-04-22T15:16:35.572928967Z Wrote inspect data to must-gather.
2026-04-22T15:16:35.584822397Z INFO: 'previous-logs' folder not found on ip-10-0-136-45.ec2.internal, skipping...
2026-04-22T15:16:35.594381260Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T15:16:35.594381260Z See 'oc rsync -h' for help and examples
2026-04-22T15:16:35.595093868Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T15:16:35.602157529Z Wrote inspect data to must-gather.
2026-04-22T15:16:35.665540536Z Wrote inspect data to must-gather.
2026-04-22T15:16:35.827894367Z Wrote inspect data to must-gather.
2026-04-22T15:16:35.854902115Z INFO: 'previous-logs' folder not found on ip-10-0-139-83.ec2.internal, skipping...
2026-04-22T15:16:35.875420139Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T15:16:35.961470433Z Wrote inspect data to must-gather.
2026-04-22T15:16:36.072038985Z Wrote inspect data to must-gather.
2026-04-22T15:16:36.131040627Z INFO: 'previous-logs' folder not found on ip-10-0-142-195.ec2.internal, skipping...
2026-04-22T15:16:36.131162352Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T15:16:36.131185872Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T15:16:36.226915602Z Wrote inspect data to must-gather.
2026-04-22T15:16:36.234132363Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T15:16:36.291456916Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T15:16:36.491629489Z Wrote inspect data to must-gather.
2026-04-22T15:16:36.560199761Z INFO: INTERCONNECT MODE
2026-04-22T15:16:36.560236196Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T15:16:36.614573097Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T15:16:36.726797939Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T15:16:36.733575499Z Wrote inspect data to must-gather.
2026-04-22T15:16:36.734444698Z INFO: Gathering OVN_Northbound from ovnkube-node-pbtw8...
2026-04-22T15:16:36.734718134Z INFO: Gathering OVN_Northbound from ovnkube-node-t88vk...
2026-04-22T15:16:36.735012800Z INFO: Gathering OVN_Northbound from ovnkube-node-x7g6p...
2026-04-22T15:16:36.735303186Z INFO: Gathering OVN_Southbound from ovnkube-node-pbtw8...
2026-04-22T15:16:36.735560954Z INFO: Gathering OVN_Southbound from ovnkube-node-t88vk...
2026-04-22T15:16:36.735885508Z INFO: Gathering OVN_Southbound from ovnkube-node-x7g6p...
2026-04-22T15:16:36.997264535Z tar: Removing leading `/' from member names
2026-04-22T15:16:37.040211415Z tar: Removing leading `/' from member names
2026-04-22T15:16:37.076599146Z tar: Removing leading `/' from member names
2026-04-22T15:16:37.091681625Z tar: Removing leading `/' from member names
2026-04-22T15:16:37.091733661Z tar: Removing leading `/' from member names
2026-04-22T15:16:37.091776447Z tar: Removing leading `/' from member names
2026-04-22T15:16:37.096486422Z Wrote inspect data to must-gather.
2026-04-22T15:16:37.158437907Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T15:16:37.350658933Z W0422 15:16:37.350612    1314 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-22T15:16:37.356362517Z Wrote inspect data to must-gather.
2026-04-22T15:16:37.384061418Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T15:16:37.460835376Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T15:16:37.599435937Z Gathering data for ns/openshift-network-console...
2026-04-22T15:16:37.611082712Z INFO: Waiting for network log collection to complete ...
2026-04-22T15:16:37.611206209Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T15:16:37.611612301Z INFO: Copying ovnk databases complete.
2026-04-22T15:16:37.613592660Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-04-22T15:16:37.616678895Z ovnk_database_store/
2026-04-22T15:16:37.616678895Z ovnk_database_store/ovnkube-node-pbtw8_nbdb
2026-04-22T15:16:37.630068367Z ovnk_database_store/ovnkube-node-t88vk_nbdb
2026-04-22T15:16:37.640873770Z ovnk_database_store/ovnkube-node-pbtw8_sbdb
2026-04-22T15:16:37.665185550Z ovnk_database_store/ovnkube-node-t88vk_sbdb
2026-04-22T15:16:37.680129704Z Wrote inspect data to must-gather.
2026-04-22T15:16:37.692151379Z ovnk_database_store/ovnkube-node-x7g6p_nbdb
2026-04-22T15:16:37.705571484Z ovnk_database_store/ovnkube-node-x7g6p_sbdb
2026-04-22T15:16:37.731483476Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T15:16:37.733989666Z INFO: Network log collection complete.
2026-04-22T15:16:37.957470236Z Wrote inspect data to must-gather.
2026-04-22T15:16:38.129793170Z Gathering data for ns/openshift-console-operator...
2026-04-22T15:16:38.144128585Z INFO: Getting status from alertmanager-main-0
2026-04-22T15:16:38.263474034Z Wrote inspect data to must-gather.
2026-04-22T15:16:38.531709413Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T15:16:38.534253771Z INFO: Fallback to identify the container image from release info
2026-04-22T15:16:38.540971221Z Gathering data for ns/openshift-console...
2026-04-22T15:16:38.993035402Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T15:16:39.070896780Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T15:16:39.194619657Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T15:16:39.331770169Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T15:16:39.401102958Z Gathering data for ns/openshift-dns-operator...
2026-04-22T15:16:39.726996803Z Gathering data for ns/openshift-dns...
2026-04-22T15:16:40.361015347Z Gathering data for ns/openshift-image-registry...
2026-04-22T15:16:40.452087743Z Waiting for performance profile collector pods to become ready: 2
2026-04-22T15:16:40.561579831Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-04-22T15:16:40.921596472Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T15:16:41.267367795Z Gathering data for ns/openshift-ingress...
2026-04-22T15:16:41.584605544Z Waiting for performance profile collector pods to become ready: 3
2026-04-22T15:16:41.665354691Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T15:16:42.134884813Z Gathering data for ns/openshift-insights...
2026-04-22T15:16:42.720456424Z Waiting for performance profile collector pods to become ready: 4
2026-04-22T15:16:43.851902042Z Waiting for performance profile collector pods to become ready: 5
2026-04-22T15:16:44.553533894Z Gathering data for ns/kserve...
2026-04-22T15:16:44.985848792Z Waiting for performance profile collector pods to become ready: 6
2026-04-22T15:16:45.566552397Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-04-22T15:16:46.106611621Z Waiting for performance profile collector pods to become ready: 7
2026-04-22T15:16:46.364074557Z Gathering data for ns/openshift-config...
2026-04-22T15:16:46.681741446Z Gathering data for ns/openshift-config-managed...
2026-04-22T15:16:47.123470350Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T15:16:47.250860069Z Waiting for performance profile collector pods to become ready: 8
2026-04-22T15:16:47.469203619Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T15:16:47.737606958Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T15:16:48.027152068Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T15:16:48.326903417Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T15:16:48.389927060Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T15:16:48.676524523Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T15:16:48.702161856Z Collecting performance related data for node ip-10-0-139-83.ec2.internal
2026-04-22T15:16:48.712252239Z Collecting performance related data for node ip-10-0-136-45.ec2.internal
2026-04-22T15:16:48.722917934Z Collecting performance related data for node ip-10-0-142-195.ec2.internal
2026-04-22T15:16:49.027998615Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T15:16:49.477311385Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T15:16:49.899934110Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T15:16:50.511183648Z Gathering data for ns/openshift-multus...
2026-04-22T15:16:50.571154620Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-04-22T15:16:52.081026740Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T15:16:53.739578472Z Gathering data for ns/openshift-host-network...
2026-04-22T15:16:54.075817059Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T15:16:54.666923982Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T15:16:55.151746569Z Gathering data for ns/openshift-network-operator...
2026-04-22T15:16:55.560971056Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T15:16:55.576797313Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-04-22T15:16:55.845544919Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T15:16:56.314329407Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T15:16:56.634706460Z Gathering data for ns/openshift-apiserver...
2026-04-22T15:16:57.056855562Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T15:16:57.335964696Z Gathering data for ns/openshift-controller-manager...
2026-04-22T15:16:57.348314532Z Collecting kubelet logs for node ip-10-0-139-83.ec2.internal
2026-04-22T15:16:57.350516267Z Collecting kubelet logs for node ip-10-0-136-45.ec2.internal
2026-04-22T15:16:57.354261700Z Collecting kubelet logs for node ip-10-0-142-195.ec2.internal
2026-04-22T15:16:57.691455938Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T15:16:57.738061855Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T15:16:57.743982136Z INFO: Node performance data collection complete.
2026-04-22T15:16:58.350969654Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T15:16:58.683002137Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T15:16:58.999330643Z Gathering data for ns/openshift-service-ca...
2026-04-22T15:16:59.359330057Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T15:17:00.297581367Z Wrote inspect data to must-gather.
2026-04-22T15:17:00.297943023Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T15:17:00.297943023Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T15:17:00.426859390Z error: the server doesn't have a resource type "clusters"
2026-04-22T15:17:00.581782927Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-04-22T15:17:00.671492825Z Caches written to disk