2026-04-16T18:58:29.512595962Z [disk usage checker] Started
2026-04-16T18:58:29.516047396Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T18:58:29.728597414Z Gathering data for ns/openshift-cluster-version...
2026-04-16T18:58:29.859815267Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T18:58:29.953003625Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T18:58:30.032741802Z Gathering data for ns/default...
2026-04-16T18:58:30.333528638Z Gathering data for ns/openshift...
2026-04-16T18:58:30.751464453Z Gathering data for ns/kube-system...
2026-04-16T18:58:31.473329053Z Gathering data for ns/openshift-etcd...
2026-04-16T18:58:31.819277525Z Wrote inspect data to must-gather.
2026-04-16T18:58:31.819310078Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T18:58:31.819310078Z     namespaces "assisted-installer" not found
2026-04-16T18:58:31.986310164Z Waiting on subprocesses to finish execution.
2026-04-16T18:58:31.996407723Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T18:58:31.997139204Z INFO: Gathering HAProxy config files
2026-04-16T18:58:31.997911766Z INFO: Collecting host service logs for crio
2026-04-16T18:58:31.998229232Z INFO: Collecting host service logs for kubelet
2026-04-16T18:58:31.998537730Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T18:58:31.998880434Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T18:58:31.999155891Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T18:58:31.999172888Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T18:58:31.999638801Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T18:58:32.000095413Z INFO: Collecting host service logs for NetworkManager
2026-04-16T18:58:32.000392580Z INFO: Collecting host service logs for openvswitch
2026-04-16T18:58:32.000706681Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T18:58:32.001020017Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T18:58:32.001317860Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T18:58:32.001811541Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T18:58:32.004162154Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T18:58:32.161509821Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T18:58:32.609352380Z No resources found in openshift-etcd namespace.
2026-04-16T18:58:32.643572531Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T18:58:32.652676170Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T18:58:32.652744756Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T18:58:33.122963412Z ERROR: No running kube-apiserver pods found
2026-04-16T18:58:33.143903007Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T18:58:33.188098473Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T18:58:33.258039120Z Wrote inspect data to must-gather.
2026-04-16T18:58:33.285707539Z No resources found
2026-04-16T18:58:33.391066090Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T18:58:33.439390227Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-rxktg
2026-04-16T18:58:33.439453142Z insights-runtime-extractor-8pgh6
2026-04-16T18:58:33.439472612Z insights-runtime-extractor-d9r4d
2026-04-16T18:58:33.439491982Z insights-runtime-extractor-v2wwf
2026-04-16T18:58:33.557687228Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T18:58:33.678623349Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T18:58:33.762368992Z INFO: Worker host service log collection to complete.
2026-04-16T18:58:33.935310714Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T18:58:33.935310714Z See 'oc rsync -h' for help and examples
2026-04-16T18:58:33.984138013Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T18:58:34.006471580Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T18:58:34.034374364Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.096345364Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.292668057Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.294751078Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T18:58:34.342630367Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.365026326Z No resources found
2026-04-16T18:58:34.421109078Z INFO: 'previous-logs' folder not found on ip-10-0-128-95.ec2.internal, skipping...
2026-04-16T18:58:34.437796472Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.452041461Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T18:58:34.453700502Z Gathering data for ns/openshift-monitoring...
2026-04-16T18:58:34.505326928Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.526635970Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T18:58:34.686062331Z tar: Removing leading `/' from member names
2026-04-16T18:58:34.697282668Z INFO: HAProxy config collection complete.
2026-04-16T18:58:34.697935670Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T18:58:34.719334244Z INFO: 'previous-logs' folder not found on ip-10-0-137-102.ec2.internal, skipping...
2026-04-16T18:58:34.744274194Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.832310682Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T18:58:34.927318947Z Wrote inspect data to must-gather.
2026-04-16T18:58:34.997288134Z INFO: 'previous-logs' folder not found on ip-10-0-143-51.ec2.internal, skipping...
2026-04-16T18:58:34.997320943Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T18:58:34.997357961Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T18:58:35.029888285Z Wrote inspect data to must-gather.
2026-04-16T18:58:35.148055500Z Wrote inspect data to must-gather.
2026-04-16T18:58:35.261506730Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T18:58:35.274517476Z Wrote inspect data to must-gather.
2026-04-16T18:58:35.323407180Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T18:58:35.544704012Z Wrote inspect data to must-gather.
2026-04-16T18:58:35.589352524Z INFO: INTERCONNECT MODE
2026-04-16T18:58:35.589385252Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T18:58:35.670541270Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T18:58:35.743521544Z INFO: Gathering OVN_Northbound from ovnkube-node-4qpjc...
2026-04-16T18:58:35.743861601Z INFO: Gathering OVN_Northbound from ovnkube-node-k7ztx...
2026-04-16T18:58:35.744071010Z INFO: Gathering OVN_Northbound from ovnkube-node-zkbc8...
2026-04-16T18:58:35.744309133Z INFO: Gathering OVN_Southbound from ovnkube-node-4qpjc...
2026-04-16T18:58:35.744548754Z INFO: Gathering OVN_Southbound from ovnkube-node-k7ztx...
2026-04-16T18:58:35.744896835Z INFO: Gathering OVN_Southbound from ovnkube-node-zkbc8...
2026-04-16T18:58:35.770773375Z Wrote inspect data to must-gather.
2026-04-16T18:58:35.984265726Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T18:58:36.029536259Z tar: Removing leading `/' from member names
2026-04-16T18:58:36.042312609Z tar: Removing leading `/' from member names
2026-04-16T18:58:36.069355523Z tar: Removing leading `/' from member names
2026-04-16T18:58:36.071280829Z tar: Removing leading `/' from member names
2026-04-16T18:58:36.073666515Z tar: Removing leading `/' from member names
2026-04-16T18:58:36.080199819Z tar: Removing leading `/' from member names
2026-04-16T18:58:36.129390378Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T18:58:36.154754697Z Wrote inspect data to must-gather.
2026-04-16T18:58:36.268402798Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T18:58:36.402059132Z Wrote inspect data to must-gather.
2026-04-16T18:58:36.550557698Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T18:58:36.625162762Z INFO: Waiting for network log collection to complete ...
2026-04-16T18:58:36.625277296Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T18:58:36.625343666Z Wrote inspect data to must-gather.
2026-04-16T18:58:36.625813691Z INFO: Copying ovnk databases complete.
2026-04-16T18:58:36.627648112Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-16T18:58:36.629940921Z ovnk_database_store/
2026-04-16T18:58:36.629940921Z ovnk_database_store/ovnkube-node-zkbc8_nbdb
2026-04-16T18:58:36.645488105Z ovnk_database_store/ovnkube-node-4qpjc_nbdb
2026-04-16T18:58:36.654695076Z ovnk_database_store/ovnkube-node-k7ztx_sbdb
2026-04-16T18:58:36.678397818Z ovnk_database_store/ovnkube-node-k7ztx_nbdb
2026-04-16T18:58:36.690291638Z ovnk_database_store/ovnkube-node-4qpjc_sbdb
2026-04-16T18:58:36.715461202Z ovnk_database_store/ovnkube-node-zkbc8_sbdb
2026-04-16T18:58:36.742562402Z INFO: Network log collection complete.
2026-04-16T18:58:36.757349552Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T18:58:36.929782555Z Wrote inspect data to must-gather.
2026-04-16T18:58:36.959650344Z Gathering data for ns/openshift-network-console...
2026-04-16T18:58:36.964106220Z INFO: Getting status from alertmanager-main-0
2026-04-16T18:58:37.208234499Z Wrote inspect data to must-gather.
2026-04-16T18:58:37.479283014Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T18:58:37.481914731Z INFO: Fallback to identify the container image from release info
2026-04-16T18:58:37.495259989Z Gathering data for ns/openshift-console-operator...
2026-04-16T18:58:37.879075195Z Gathering data for ns/openshift-console...
2026-04-16T18:58:38.056208006Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T18:58:38.181841738Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T18:58:38.328443784Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T18:58:38.367386890Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T18:58:38.760003662Z Gathering data for ns/openshift-dns-operator...
2026-04-16T18:58:39.082845443Z Gathering data for ns/openshift-dns...
2026-04-16T18:58:39.463371559Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T18:58:39.540266814Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T18:58:39.799370892Z Gathering data for ns/openshift-image-registry...
2026-04-16T18:58:40.313888700Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T18:58:40.605715082Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T18:58:40.672404596Z Gathering data for ns/openshift-ingress...
2026-04-16T18:58:41.018570530Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T18:58:41.419835180Z Gathering data for ns/openshift-insights...
2026-04-16T18:58:41.722926618Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T18:58:42.844104232Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T18:58:43.572244508Z Gathering data for ns/kserve...
2026-04-16T18:58:43.999212471Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T18:58:44.547177720Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T18:58:45.123180942Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T18:58:45.468138122Z Gathering data for ns/openshift-config...
2026-04-16T18:58:45.785036011Z Gathering data for ns/openshift-config-managed...
2026-04-16T18:58:46.210053459Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T18:58:46.261350768Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T18:58:46.530761868Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T18:58:46.541680939Z Collecting performance related data for node ip-10-0-137-102.ec2.internal
2026-04-16T18:58:46.548804006Z Collecting performance related data for node ip-10-0-143-51.ec2.internal
2026-04-16T18:58:46.557417558Z Collecting performance related data for node ip-10-0-128-95.ec2.internal
2026-04-16T18:58:46.981854146Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T18:58:47.478058808Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T18:58:47.890301233Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T18:58:48.249140156Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T18:58:48.595541402Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T18:58:48.963027884Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T18:58:49.329177283Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T18:58:49.554877967Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T18:58:49.953529166Z Gathering data for ns/openshift-multus...
2026-04-16T18:58:51.728924890Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T18:58:53.471611831Z Gathering data for ns/openshift-host-network...
2026-04-16T18:58:53.792023064Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T18:58:54.382701885Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T18:58:54.560802872Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T18:58:54.824450722Z Gathering data for ns/openshift-network-operator...
2026-04-16T18:58:54.893548101Z Collecting kubelet logs for node ip-10-0-137-102.ec2.internal
2026-04-16T18:58:54.895426564Z Collecting kubelet logs for node ip-10-0-143-51.ec2.internal
2026-04-16T18:58:54.897271832Z Collecting kubelet logs for node ip-10-0-128-95.ec2.internal
2026-04-16T18:58:55.280362199Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T18:58:55.290828500Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T18:58:55.296823581Z INFO: Node performance data collection complete.
2026-04-16T18:58:55.608601760Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T18:58:56.078209666Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T18:58:56.398322044Z Gathering data for ns/openshift-apiserver...
2026-04-16T18:58:56.789917792Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T18:58:57.067737088Z Gathering data for ns/openshift-controller-manager...
2026-04-16T18:58:57.343788324Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T18:58:57.986561141Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T18:58:58.319487516Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T18:58:58.641847143Z Gathering data for ns/openshift-service-ca...
2026-04-16T18:58:59.017985644Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T18:58:59.567233992Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T18:58:59.915134864Z Wrote inspect data to must-gather.
2026-04-16T18:58:59.915175889Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T18:58:59.915175889Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T18:59:00.056285296Z error: the server doesn't have a resource type "clusters"
2026-04-16T18:59:00.232383001Z Caches written to disk