2026-05-28T10:33:46.596444939Z [disk usage checker] Started
2026-05-28T10:33:46.599774891Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-05-28T10:33:46.826181599Z Gathering data for ns/openshift-cluster-version...
2026-05-28T10:33:46.952039977Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T10:33:47.052402753Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T10:33:47.135267303Z Gathering data for ns/default...
2026-05-28T10:33:47.440552351Z Gathering data for ns/openshift...
2026-05-28T10:33:47.853274190Z Gathering data for ns/kube-system...
2026-05-28T10:33:48.596640237Z Gathering data for ns/openshift-etcd...
2026-05-28T10:33:48.803332812Z Waiting on subprocesses to finish execution.
2026-05-28T10:33:48.809212064Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-28T10:33:48.809568714Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-28T10:33:48.814120818Z INFO: Collecting host service logs for crio
2026-05-28T10:33:48.814462452Z INFO: Collecting host service logs for kubelet
2026-05-28T10:33:48.814786651Z INFO: Collecting host service logs for rpm-ostreed
2026-05-28T10:33:48.815242427Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-28T10:33:48.815614325Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-28T10:33:48.815961513Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-28T10:33:48.816304159Z INFO: Collecting host service logs for NetworkManager
2026-05-28T10:33:48.816637246Z INFO: Collecting host service logs for openvswitch
2026-05-28T10:33:48.816950181Z INFO: Collecting host service logs for ovs-configuration
2026-05-28T10:33:48.817301710Z INFO: Collecting host service logs for ovsdb-server
2026-05-28T10:33:48.817644766Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-28T10:33:48.818049742Z INFO: Waiting for worker host service log collection to complete ...
2026-05-28T10:33:48.827239819Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-28T10:33:48.829098273Z INFO: Gathering HAProxy config files
2026-05-28T10:33:49.005059438Z INFO: Waiting for node performance related collection to complete ...
2026-05-28T10:33:49.152462059Z Wrote inspect data to must-gather.
2026-05-28T10:33:49.152508577Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T10:33:49.152508577Z     namespaces "assisted-installer" not found
2026-05-28T10:33:49.208242780Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-66dzt
2026-05-28T10:33:49.208293291Z insights-runtime-extractor-hxbg5
2026-05-28T10:33:49.208313030Z insights-runtime-extractor-pr7st
2026-05-28T10:33:49.208332390Z insights-runtime-extractor-zgnjq
2026-05-28T10:33:49.271742910Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-28T10:33:49.429351056Z No resources found in openshift-etcd namespace.
2026-05-28T10:33:49.880506853Z error: the server doesn't have a resource type "performanceprofile"
2026-05-28T10:33:49.892739537Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-28T10:33:49.892739537Z See 'oc rsync -h' for help and examples
2026-05-28T10:33:49.905262156Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-28T10:33:49.951761074Z No resources found
2026-05-28T10:33:49.956711828Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-28T10:33:49.956747836Z INFO: on-disk MachineConfig config collection complete.
2026-05-28T10:33:49.974303383Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-28T10:33:50.005289707Z Wrote inspect data to must-gather.
2026-05-28T10:33:50.060426040Z INFO: Worker host service log collection to complete.
2026-05-28T10:33:50.073030216Z INFO: "metallb-operator" not detected. Skipping.
2026-05-28T10:33:50.083205008Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-28T10:33:50.176395490Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-28T10:33:50.332711931Z Wrote inspect data to must-gather.
2026-05-28T10:33:50.555424016Z ERROR: No running kube-apiserver pods found
2026-05-28T10:33:50.660437077Z Wrote inspect data to must-gather.
2026-05-28T10:33:50.703661106Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T10:33:50.710550269Z INFO: 'previous-logs' folder not found on ip-10-0-128-206.ec2.internal, skipping...
2026-05-28T10:33:50.759663408Z Wrote inspect data to must-gather.
2026-05-28T10:33:50.768457040Z Wrote inspect data to must-gather.
2026-05-28T10:33:50.789159251Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-28T10:33:50.859555148Z Gathering data for ns/openshift-monitoring...
2026-05-28T10:33:50.880048443Z tar: Removing leading `/' from member names
2026-05-28T10:33:50.880805886Z No resources found
2026-05-28T10:33:50.890397130Z INFO: HAProxy config collection complete.
2026-05-28T10:33:50.914286697Z Wrote inspect data to must-gather.
2026-05-28T10:33:50.958852350Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-28T10:33:51.016852397Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.037526699Z error: the server doesn't have a resource type "machineconfigs"
2026-05-28T10:33:51.119168594Z INFO: 'previous-logs' folder not found on ip-10-0-129-122.ec2.internal, skipping...
2026-05-28T10:33:51.119318667Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.233397074Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.331780224Z INFO: Getting rules from prometheus-k8s-0
2026-05-28T10:33:51.360644615Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.364445188Z INFO: 'previous-logs' folder not found on ip-10-0-133-103.ec2.internal, skipping...
2026-05-28T10:33:51.364487606Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-28T10:33:51.364517867Z INFO: Machine Config Daemon termination log collection complete.
2026-05-28T10:33:51.444113386Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.570763252Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.575243555Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-28T10:33:51.604561280Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-05-28T10:33:51.762135218Z INFO: Getting status/config from prometheus-k8s-0
2026-05-28T10:33:51.783613499Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.849200215Z INFO: INTERCONNECT MODE
2026-05-28T10:33:51.849313665Z INFO: Gathering ovn-kubernetes DBs
2026-05-28T10:33:51.965024581Z Wrote inspect data to must-gather.
2026-05-28T10:33:51.989142416Z INFO: Gathering OVN_Northbound from ovnkube-node-2lzsn...
2026-05-28T10:33:51.989420006Z INFO: Gathering OVN_Northbound from ovnkube-node-j4whq...
2026-05-28T10:33:51.989655706Z INFO: Gathering OVN_Northbound from ovnkube-node-xmh9c...
2026-05-28T10:33:51.989925380Z INFO: Gathering OVN_Southbound from ovnkube-node-2lzsn...
2026-05-28T10:33:51.990180473Z INFO: Gathering OVN_Southbound from ovnkube-node-j4whq...
2026-05-28T10:33:51.990441353Z INFO: Gathering OVN_Southbound from ovnkube-node-xmh9c...
2026-05-28T10:33:52.264545559Z Wrote inspect data to must-gather.
2026-05-28T10:33:52.310233964Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-28T10:33:52.331202745Z tar: Removing leading `/' from member names
2026-05-28T10:33:52.336387122Z tar: Removing leading `/' from member names
2026-05-28T10:33:52.341218892Z tar: Removing leading `/' from member names
2026-05-28T10:33:52.359318394Z tar: Removing leading `/' from member names
2026-05-28T10:33:52.398499108Z tar: Removing leading `/' from member names
2026-05-28T10:33:52.406396885Z tar: Removing leading `/' from member names
2026-05-28T10:33:52.475435994Z Wrote inspect data to must-gather.
2026-05-28T10:33:52.557783712Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-28T10:33:52.654124242Z Wrote inspect data to must-gather.
2026-05-28T10:33:52.684935792Z INFO: Waiting for network log collection to complete ...
2026-05-28T10:33:52.684964254Z INFO: Waiting for ovnk database copies to complete ...
2026-05-28T10:33:52.685354052Z INFO: Copying ovnk databases complete.
2026-05-28T10:33:52.686819888Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-05-28T10:33:52.689289069Z ovnk_database_store/
2026-05-28T10:33:52.689317858Z ovnk_database_store/ovnkube-node-2lzsn_nbdb
2026-05-28T10:33:52.697877746Z ovnk_database_store/ovnkube-node-j4whq_sbdb
2026-05-28T10:33:52.718107755Z ovnk_database_store/ovnkube-node-2lzsn_sbdb
2026-05-28T10:33:52.737490701Z ovnk_database_store/ovnkube-node-j4whq_nbdb
2026-05-28T10:33:52.745925816Z ovnk_database_store/ovnkube-node-xmh9c_sbdb
2026-05-28T10:33:52.767195372Z ovnk_database_store/ovnkube-node-xmh9c_nbdb
2026-05-28T10:33:52.777475084Z INFO: Network log collection complete.
2026-05-28T10:33:52.818595521Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-28T10:33:52.826819988Z Wrote inspect data to must-gather.
2026-05-28T10:33:52.996203901Z Wrote inspect data to must-gather.
2026-05-28T10:33:53.087976617Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-28T10:33:53.229120526Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-28T10:33:53.231799104Z INFO: Fallback to identify the container image from release info
2026-05-28T10:33:53.264168168Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T10:33:53.455374481Z INFO: Getting status from alertmanager-main-0
2026-05-28T10:33:53.713953628Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-28T10:33:53.801702756Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T10:33:53.826978691Z daemonset.apps/perf-node-gather-daemonset created
2026-05-28T10:33:53.944957373Z Waiting for performance profile collector pods to become ready: 1
2026-05-28T10:33:53.968838702Z Gathering data for ns/openshift-network-console...
2026-05-28T10:33:54.361108697Z Gathering data for ns/openshift-console-operator...
2026-05-28T10:33:54.725776870Z Gathering data for ns/openshift-console...
2026-05-28T10:33:55.055134717Z Waiting for performance profile collector pods to become ready: 2
2026-05-28T10:33:55.144762422Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-28T10:33:55.499076266Z Gathering data for ns/openshift-dns-operator...
2026-05-28T10:33:55.803063061Z Gathering data for ns/openshift-dns...
2026-05-28T10:33:56.172873693Z Waiting for performance profile collector pods to become ready: 3
2026-05-28T10:33:56.412535097Z Gathering data for ns/openshift-image-registry...
2026-05-28T10:33:56.609590587Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-05-28T10:33:56.929433797Z Gathering data for ns/openshift-ingress-operator...
2026-05-28T10:33:57.276843033Z Gathering data for ns/openshift-ingress...
2026-05-28T10:33:57.285904742Z Waiting for performance profile collector pods to become ready: 4
2026-05-28T10:33:57.594296484Z Gathering data for ns/openshift-ingress-canary...
2026-05-28T10:33:58.020875296Z Gathering data for ns/openshift-insights...
2026-05-28T10:33:58.383473141Z Waiting for performance profile collector pods to become ready: 5
2026-05-28T10:33:59.489155000Z Waiting for performance profile collector pods to become ready: 6
2026-05-28T10:34:00.086187774Z Gathering data for ns/openshift-config...
2026-05-28T10:34:00.395727823Z Gathering data for ns/openshift-config-managed...
2026-05-28T10:34:00.592166432Z Waiting for performance profile collector pods to become ready: 7
2026-05-28T10:34:00.817217373Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-28T10:34:01.068591413Z Gathering data for ns/openshift-kube-apiserver...
2026-05-28T10:34:01.341338254Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-28T10:34:01.614739359Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-05-28T10:34:01.689440065Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-28T10:34:01.714488751Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-28T10:34:01.969069928Z Collecting performance related data for node ip-10-0-129-122.ec2.internal
2026-05-28T10:34:01.978395510Z Collecting performance related data for node ip-10-0-128-206.ec2.internal
2026-05-28T10:34:01.985227926Z Collecting performance related data for node ip-10-0-133-103.ec2.internal
2026-05-28T10:34:02.033869286Z Gathering data for ns/openshift-kube-scheduler...
2026-05-28T10:34:02.426111829Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-28T10:34:02.842869335Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-28T10:34:03.210650464Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-28T10:34:03.589556983Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-28T10:34:04.103558433Z Gathering data for ns/openshift-multus...
2026-05-28T10:34:05.794056582Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-28T10:34:06.620865103Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-05-28T10:34:07.406122690Z Gathering data for ns/openshift-host-network...
2026-05-28T10:34:07.726728148Z Gathering data for ns/openshift-network-diagnostics...
2026-05-28T10:34:08.279256393Z Gathering data for ns/openshift-network-node-identity...
2026-05-28T10:34:08.729083047Z Gathering data for ns/openshift-network-operator...
2026-05-28T10:34:09.133593854Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-28T10:34:09.402300125Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-28T10:34:09.930578837Z Gathering data for ns/openshift-apiserver-operator...
2026-05-28T10:34:10.123440087Z Collecting kubelet logs for node ip-10-0-129-122.ec2.internal
2026-05-28T10:34:10.126329046Z Collecting kubelet logs for node ip-10-0-128-206.ec2.internal
2026-05-28T10:34:10.128495368Z Collecting kubelet logs for node ip-10-0-133-103.ec2.internal
2026-05-28T10:34:10.258087647Z Gathering data for ns/openshift-apiserver...
2026-05-28T10:34:10.495067414Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-28T10:34:10.500801109Z INFO: Node performance data collection complete.
2026-05-28T10:34:10.692423255Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-28T10:34:10.974372559Z Gathering data for ns/openshift-controller-manager...
2026-05-28T10:34:11.288316471Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-28T10:34:11.625326851Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-05-28T10:34:11.940790017Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-28T10:34:12.245087747Z Gathering data for ns/openshift-service-ca-operator...
2026-05-28T10:34:12.538641701Z Gathering data for ns/openshift-service-ca...
2026-05-28T10:34:12.950961134Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-28T10:34:14.025896817Z Wrote inspect data to must-gather.
2026-05-28T10:34:14.025943361Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T10:34:14.025943361Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-28T10:34:14.146709370Z error: the server doesn't have a resource type "clusters"
2026-05-28T10:34:14.279996191Z Caches written to disk