2026-04-16T21:13:53.831951266Z [disk usage checker] Started
2026-04-16T21:13:53.867054284Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:13:54.057568744Z Gathering data for ns/openshift-cluster-version...
2026-04-16T21:13:54.226074653Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T21:13:54.321988245Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T21:13:54.417768869Z Gathering data for ns/default...
2026-04-16T21:13:54.745758699Z Gathering data for ns/openshift...
2026-04-16T21:13:55.197823325Z Gathering data for ns/kube-system...
2026-04-16T21:13:55.940255384Z Gathering data for ns/openshift-etcd...
2026-04-16T21:13:56.272360602Z Waiting on subprocesses to finish execution.
2026-04-16T21:13:56.275843196Z INFO: Gathering HAProxy config files
2026-04-16T21:13:56.301885540Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T21:13:56.311528296Z Wrote inspect data to must-gather.
2026-04-16T21:13:56.311644358Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T21:13:56.311644358Z     namespaces "assisted-installer" not found
2026-04-16T21:13:56.313225605Z INFO: Collecting host service logs for crio
2026-04-16T21:13:56.313851257Z INFO: Collecting host service logs for kubelet
2026-04-16T21:13:56.314220645Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T21:13:56.314558053Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T21:13:56.314922682Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T21:13:56.315278772Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T21:13:56.315706756Z INFO: Collecting host service logs for NetworkManager
2026-04-16T21:13:56.316126101Z INFO: Collecting host service logs for openvswitch
2026-04-16T21:13:56.316461387Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T21:13:56.316821836Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T21:13:56.317165347Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T21:13:56.317582087Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T21:13:56.324266824Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T21:13:56.333456455Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T21:13:56.357122142Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T21:13:57.047221076Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T21:13:57.102172981Z No resources found in openshift-etcd namespace.
2026-04-16T21:13:57.128904072Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T21:13:57.134955265Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-x6rb4
2026-04-16T21:13:57.134992970Z insights-runtime-extractor-fkfvs
2026-04-16T21:13:57.135012543Z insights-runtime-extractor-s5xct
2026-04-16T21:13:57.135033103Z insights-runtime-extractor-scchg
2026-04-16T21:13:57.398809155Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T21:13:57.398836083Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T21:13:57.437166852Z No resources found
2026-04-16T21:13:57.679241015Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T21:13:57.700021360Z Wrote inspect data to must-gather.
2026-04-16T21:13:57.752866395Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T21:13:57.752866395Z See 'oc rsync -h' for help and examples
2026-04-16T21:13:57.760973846Z INFO: Worker host service log collection to complete.
2026-04-16T21:13:57.801262559Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T21:13:57.801844038Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T21:13:57.850114663Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T21:13:57.948587885Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T21:13:58.079787000Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T21:13:58.127751354Z INFO: 'previous-logs' folder not found on ip-10-0-129-199.ec2.internal, skipping...
2026-04-16T21:13:58.166031436Z ERROR: No running kube-apiserver pods found
2026-04-16T21:13:58.759629554Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T21:13:58.804432695Z No resources found
2026-04-16T21:13:58.886082094Z Wrote inspect data to must-gather.
2026-04-16T21:13:58.910768638Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:13:58.961810316Z Wrote inspect data to must-gather.
2026-04-16T21:13:59.021679456Z Gathering data for ns/kuadrant-system...
2026-04-16T21:13:59.049339855Z INFO: 'previous-logs' folder not found on ip-10-0-134-79.ec2.internal, skipping...
2026-04-16T21:13:59.051174458Z INFO: HAProxy config collection complete.
2026-04-16T21:13:59.266350387Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T21:13:59.309689890Z Wrote inspect data to must-gather.
2026-04-16T21:13:59.313330661Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T21:13:59.360555630Z Wrote inspect data to must-gather.
2026-04-16T21:13:59.386548365Z Wrote inspect data to must-gather.
2026-04-16T21:13:59.600337298Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T21:13:59.613173602Z Wrote inspect data to must-gather.
2026-04-16T21:13:59.642350699Z INFO: 'previous-logs' folder not found on ip-10-0-142-90.ec2.internal, skipping...
2026-04-16T21:13:59.642457231Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T21:13:59.642509469Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T21:13:59.835647414Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T21:13:59.892423785Z Wrote inspect data to must-gather.
2026-04-16T21:13:59.958859466Z Wrote inspect data to must-gather.
2026-04-16T21:14:00.206886939Z Wrote inspect data to must-gather.
2026-04-16T21:14:00.291683428Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T21:14:00.318145828Z Wrote inspect data to must-gather.
2026-04-16T21:14:00.328507387Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T21:14:00.451529681Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T21:14:00.626576093Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T21:14:00.723078398Z INFO: INTERCONNECT MODE
2026-04-16T21:14:00.723164290Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T21:14:00.761065739Z Wrote inspect data to must-gather.
2026-04-16T21:14:00.935788400Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T21:14:00.944264903Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T21:14:00.956548815Z INFO: Gathering OVN_Northbound from ovnkube-node-5qqqk...
2026-04-16T21:14:00.956876890Z INFO: Gathering OVN_Northbound from ovnkube-node-jx6zj...
2026-04-16T21:14:00.957145008Z INFO: Gathering OVN_Northbound from ovnkube-node-sj6sh...
2026-04-16T21:14:00.957459298Z INFO: Gathering OVN_Southbound from ovnkube-node-5qqqk...
2026-04-16T21:14:00.957750653Z INFO: Gathering OVN_Southbound from ovnkube-node-jx6zj...
2026-04-16T21:14:00.958024482Z INFO: Gathering OVN_Southbound from ovnkube-node-sj6sh...
2026-04-16T21:14:01.276364743Z Wrote inspect data to must-gather.
2026-04-16T21:14:01.314848612Z tar: Removing leading `/' from member names
2026-04-16T21:14:01.349583805Z tar: Removing leading `/' from member names
2026-04-16T21:14:01.378653497Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T21:14:01.389778829Z Gathering data for ns/openshift-monitoring...
2026-04-16T21:14:01.448137013Z tar: Removing leading `/' from member names
2026-04-16T21:14:01.456496190Z tar: Removing leading `/' from member names
2026-04-16T21:14:01.490701103Z tar: Removing leading `/' from member names
2026-04-16T21:14:01.504086781Z tar: Removing leading `/' from member names
2026-04-16T21:14:01.722802250Z Wrote inspect data to must-gather.
2026-04-16T21:14:01.923511410Z INFO: Getting status from alertmanager-main-0
2026-04-16T21:14:02.101430880Z Wrote inspect data to must-gather.
2026-04-16T21:14:02.130342626Z INFO: Waiting for network log collection to complete ...
2026-04-16T21:14:02.130368320Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T21:14:02.130773156Z INFO: Copying ovnk databases complete.
2026-04-16T21:14:02.132334258Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-04-16T21:14:02.134328802Z ovnk_database_store/
2026-04-16T21:14:02.134328802Z ovnk_database_store/ovnkube-node-jx6zj_sbdb
2026-04-16T21:14:02.164283435Z ovnk_database_store/ovnkube-node-sj6sh_nbdb
2026-04-16T21:14:02.177892455Z ovnk_database_store/ovnkube-node-jx6zj_nbdb
2026-04-16T21:14:02.190113933Z ovnk_database_store/ovnkube-node-5qqqk_nbdb
2026-04-16T21:14:02.200992979Z ovnk_database_store/ovnkube-node-5qqqk_sbdb
2026-04-16T21:14:02.230532815Z ovnk_database_store/ovnkube-node-sj6sh_sbdb
2026-04-16T21:14:02.261746911Z INFO: Network log collection complete.
2026-04-16T21:14:02.430273251Z Wrote inspect data to must-gather.
2026-04-16T21:14:02.726825738Z Wrote inspect data to must-gather.
2026-04-16T21:14:03.046500915Z Wrote inspect data to must-gather.
2026-04-16T21:14:03.307151045Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T21:14:03.310421980Z INFO: Fallback to identify the container image from release info
2026-04-16T21:14:03.633395837Z Gathering data for ns/openshift-network-console...
2026-04-16T21:14:03.931724642Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-16T21:14:04.007685640Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:14:04.094945055Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T21:14:04.165269943Z Gathering data for ns/openshift-console-operator...
2026-04-16T21:14:04.249134309Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T21:14:04.671394561Z Gathering data for ns/openshift-console...
2026-04-16T21:14:05.234121027Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T21:14:05.400857659Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T21:14:05.508756289Z Wrote inspect data to must-gather.
2026-04-16T21:14:05.652352718Z Gathering data for ns/openshift-dns-operator...
2026-04-16T21:14:06.027715262Z Gathering data for ns/openshift-dns...
2026-04-16T21:14:06.534361353Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T21:14:06.671208252Z Gathering data for ns/openshift-image-registry...
2026-04-16T21:14:07.143586585Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T21:14:07.499820089Z Gathering data for ns/openshift-ingress...
2026-04-16T21:14:07.639439867Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T21:14:08.355806971Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T21:14:08.767048849Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T21:14:08.824909933Z Gathering data for ns/openshift-insights...
2026-04-16T21:14:09.012529400Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:14:09.869567369Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T21:14:10.922238273Z Gathering data for ns/opendatahub...
2026-04-16T21:14:10.988089494Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T21:14:12.092008467Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T21:14:12.350273280Z Collecting performance related data for node ip-10-0-134-79.ec2.internal
2026-04-16T21:14:12.357069489Z Collecting performance related data for node ip-10-0-129-199.ec2.internal
2026-04-16T21:14:12.364649756Z Collecting performance related data for node ip-10-0-142-90.ec2.internal
2026-04-16T21:14:12.425041134Z Gathering data for ns/openshift-lws-operator...
2026-04-16T21:14:13.907542107Z Gathering data for ns/openshift-config...
2026-04-16T21:14:14.017635239Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:14:14.236264481Z Gathering data for ns/openshift-config-managed...
2026-04-16T21:14:14.783219166Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T21:14:15.138575362Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T21:14:15.475757847Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T21:14:15.842888568Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T21:14:16.203830157Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T21:14:16.541828129Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T21:14:16.921022928Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T21:14:17.359369240Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T21:14:17.860494971Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T21:14:18.559772850Z Gathering data for ns/openshift-multus...
2026-04-16T21:14:19.022155772Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:14:20.560454563Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T21:14:20.653982468Z Collecting kubelet logs for node ip-10-0-134-79.ec2.internal
2026-04-16T21:14:20.656168827Z Collecting kubelet logs for node ip-10-0-129-199.ec2.internal
2026-04-16T21:14:20.659828682Z Collecting kubelet logs for node ip-10-0-142-90.ec2.internal
2026-04-16T21:14:21.014032623Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T21:14:21.020384194Z INFO: Node performance data collection complete.
2026-04-16T21:14:22.251998923Z Gathering data for ns/openshift-host-network...
2026-04-16T21:14:22.617161160Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T21:14:23.191228405Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T21:14:23.667646064Z Gathering data for ns/openshift-network-operator...
2026-04-16T21:14:24.028003502Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:14:24.134910708Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T21:14:24.465410909Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T21:14:24.993259769Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T21:14:25.317482397Z Gathering data for ns/openshift-apiserver...
2026-04-16T21:14:25.800362717Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T21:14:26.091976226Z Gathering data for ns/openshift-controller-manager...
2026-04-16T21:14:26.429165011Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T21:14:27.100281425Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T21:14:27.471100351Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T21:14:27.817144089Z Gathering data for ns/openshift-service-ca...
2026-04-16T21:14:28.214282714Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T21:14:29.032865773Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T21:14:29.264755056Z Wrote inspect data to must-gather.
2026-04-16T21:14:29.264799563Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T21:14:29.264799563Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T21:14:29.390272627Z error: the server doesn't have a resource type "clusters"
2026-04-16T21:14:29.567624358Z Caches written to disk