2026-04-17T11:53:42.780541457Z [disk usage checker] Started
2026-04-17T11:53:42.813358783Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T11:53:42.981397451Z Gathering data for ns/openshift-cluster-version...
2026-04-17T11:53:43.123359423Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T11:53:43.203657738Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T11:53:43.277241517Z Gathering data for ns/default...
2026-04-17T11:53:43.585641456Z Gathering data for ns/openshift...
2026-04-17T11:53:43.980065615Z Gathering data for ns/kube-system...
2026-04-17T11:53:44.595678525Z Gathering data for ns/openshift-etcd...
2026-04-17T11:53:44.874283883Z Wrote inspect data to must-gather.
2026-04-17T11:53:44.874318028Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T11:53:44.874318028Z     namespaces "assisted-installer" not found
2026-04-17T11:53:45.033936264Z Waiting on subprocesses to finish execution.
2026-04-17T11:53:45.041431237Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-17T11:53:45.045744604Z INFO: Gathering HAProxy config files
2026-04-17T11:53:45.050942398Z INFO: Collecting host service logs for crio
2026-04-17T11:53:45.051293881Z INFO: Collecting host service logs for kubelet
2026-04-17T11:53:45.051642251Z INFO: Collecting host service logs for rpm-ostreed
2026-04-17T11:53:45.051986541Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-17T11:53:45.052345680Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-17T11:53:45.052685727Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-17T11:53:45.052727118Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-17T11:53:45.053023377Z INFO: Collecting host service logs for NetworkManager
2026-04-17T11:53:45.053343240Z INFO: Collecting host service logs for openvswitch
2026-04-17T11:53:45.053645841Z INFO: Collecting host service logs for ovs-configuration
2026-04-17T11:53:45.053939118Z INFO: Collecting host service logs for ovsdb-server
2026-04-17T11:53:45.054228557Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-17T11:53:45.054657739Z INFO: Waiting for worker host service log collection to complete ...
2026-04-17T11:53:45.060087699Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-17T11:53:45.115147259Z INFO: Waiting for node performance related collection to complete ...
2026-04-17T11:53:45.721623363Z No resources found in openshift-etcd namespace.
2026-04-17T11:53:45.823737708Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-17T11:53:45.823781747Z INFO: on-disk MachineConfig config collection complete.
2026-04-17T11:53:45.936745831Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-17T11:53:45.983803137Z Wrote inspect data to must-gather.
2026-04-17T11:53:46.012515810Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-17T11:53:46.098999649Z No resources found
2026-04-17T11:53:46.281643580Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-qvvgv
2026-04-17T11:53:46.281643580Z insights-runtime-extractor-2h4l7
2026-04-17T11:53:46.281643580Z insights-runtime-extractor-dn4vn
2026-04-17T11:53:46.281643580Z insights-runtime-extractor-xs4lh
2026-04-17T11:53:46.282383104Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-17T11:53:46.302204528Z INFO: Worker host service log collection to complete.
2026-04-17T11:53:46.342636397Z error: the server doesn't have a resource type "performanceprofile"
2026-04-17T11:53:46.377132148Z INFO: "metallb-operator" not detected. Skipping.
2026-04-17T11:53:46.585787006Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T11:53:46.608116318Z Wrote inspect data to must-gather.
2026-04-17T11:53:46.685588667Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-17T11:53:46.763759152Z Wrote inspect data to must-gather.
2026-04-17T11:53:46.771766681Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-17T11:53:46.822546556Z ERROR: No running kube-apiserver pods found
2026-04-17T11:53:46.831697750Z Wrote inspect data to must-gather.
2026-04-17T11:53:46.970897902Z No resources found
2026-04-17T11:53:46.983742027Z Gathering data for ns/openshift-monitoring...
2026-04-17T11:53:46.989747933Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-17T11:53:46.989747933Z See 'oc rsync -h' for help and examples
2026-04-17T11:53:47.000574313Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-17T11:53:47.074444280Z INFO: 'previous-logs' folder not found on ip-10-0-130-210.ec2.internal, skipping...
2026-04-17T11:53:47.090929092Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-17T11:53:47.198247911Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.199668660Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.274491891Z tar: Removing leading `/' from member names
2026-04-17T11:53:47.279880115Z INFO: HAProxy config collection complete.
2026-04-17T11:53:47.286690490Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.387025377Z error: the server doesn't have a resource type "machineconfigs"
2026-04-17T11:53:47.389947837Z INFO: 'previous-logs' folder not found on ip-10-0-134-64.ec2.internal, skipping...
2026-04-17T11:53:47.477550988Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.555443084Z INFO: Getting rules from prometheus-k8s-0
2026-04-17T11:53:47.632577677Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.658960092Z INFO: 'previous-logs' folder not found on ip-10-0-140-245.ec2.internal, skipping...
2026-04-17T11:53:47.659073322Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-17T11:53:47.659125836Z INFO: Machine Config Daemon termination log collection complete.
2026-04-17T11:53:47.722825674Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.820700297Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T11:53:47.835707817Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.929310286Z INFO: Getting status/config from prometheus-k8s-0
2026-04-17T11:53:47.946051826Z Wrote inspect data to must-gather.
2026-04-17T11:53:47.986481185Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-17T11:53:48.199786124Z Wrote inspect data to must-gather.
2026-04-17T11:53:48.227321131Z INFO: INTERCONNECT MODE
2026-04-17T11:53:48.227413424Z INFO: Gathering ovn-kubernetes DBs
2026-04-17T11:53:48.320738421Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T11:53:48.339521901Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-17T11:53:48.381579743Z INFO: Gathering OVN_Northbound from ovnkube-node-9qrz2...
2026-04-17T11:53:48.381902580Z INFO: Gathering OVN_Northbound from ovnkube-node-vjfgd...
2026-04-17T11:53:48.382166843Z INFO: Gathering OVN_Northbound from ovnkube-node-znj2s...
2026-04-17T11:53:48.382481847Z INFO: Gathering OVN_Southbound from ovnkube-node-9qrz2...
2026-04-17T11:53:48.382772773Z INFO: Gathering OVN_Southbound from ovnkube-node-vjfgd...
2026-04-17T11:53:48.383043655Z INFO: Gathering OVN_Southbound from ovnkube-node-znj2s...
2026-04-17T11:53:48.415048142Z Wrote inspect data to must-gather.
2026-04-17T11:53:48.653389836Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-17T11:53:48.693184044Z tar: Removing leading `/' from member names
2026-04-17T11:53:48.695936208Z tar: Removing leading `/' from member names
2026-04-17T11:53:48.701413727Z tar: Removing leading `/' from member names
2026-04-17T11:53:48.708490039Z tar: Removing leading `/' from member names
2026-04-17T11:53:48.734153720Z tar: Removing leading `/' from member names
2026-04-17T11:53:48.758370408Z tar: Removing leading `/' from member names
2026-04-17T11:53:48.785700994Z Wrote inspect data to must-gather.
2026-04-17T11:53:48.917349335Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-17T11:53:49.013800178Z W0417 11:53:49.013757    1287 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-17T11:53:49.019699105Z Wrote inspect data to must-gather.
2026-04-17T11:53:49.100164720Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T11:53:49.170602833Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-17T11:53:49.184048826Z INFO: Waiting for network log collection to complete ...
2026-04-17T11:53:49.184077629Z INFO: Waiting for ovnk database copies to complete ...
2026-04-17T11:53:49.184575249Z INFO: Copying ovnk databases complete.
2026-04-17T11:53:49.186550452Z 3.0M	must-gather/network_logs/ovnk_database_store
2026-04-17T11:53:49.188957366Z ovnk_database_store/
2026-04-17T11:53:49.188980286Z ovnk_database_store/ovnkube-node-9qrz2_nbdb
2026-04-17T11:53:49.199378106Z ovnk_database_store/ovnkube-node-znj2s_nbdb
2026-04-17T11:53:49.206187697Z ovnk_database_store/ovnkube-node-9qrz2_sbdb
2026-04-17T11:53:49.222643789Z ovnk_database_store/ovnkube-node-vjfgd_sbdb
2026-04-17T11:53:49.243170819Z ovnk_database_store/ovnkube-node-vjfgd_nbdb
2026-04-17T11:53:49.250476922Z ovnk_database_store/ovnkube-node-znj2s_sbdb
2026-04-17T11:53:49.272618240Z INFO: Network log collection complete.
2026-04-17T11:53:49.277771242Z Gathering data for ns/openshift-network-console...
2026-04-17T11:53:49.331226530Z Wrote inspect data to must-gather.
2026-04-17T11:53:49.508841291Z INFO: Getting status from alertmanager-main-0
2026-04-17T11:53:49.613017742Z Wrote inspect data to must-gather.
2026-04-17T11:53:49.682990248Z Gathering data for ns/openshift-console-operator...
2026-04-17T11:53:49.875091890Z Wrote inspect data to must-gather.
2026-04-17T11:53:50.037179534Z Gathering data for ns/openshift-console...
2026-04-17T11:53:50.110186865Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-17T11:53:50.112668294Z INFO: Fallback to identify the container image from release info
2026-04-17T11:53:50.433184449Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-17T11:53:50.657510857Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-17T11:53:50.745936636Z Gathering data for ns/openshift-dns-operator...
2026-04-17T11:53:50.784869810Z daemonset.apps/perf-node-gather-daemonset created
2026-04-17T11:53:50.904938017Z Waiting for performance profile collector pods to become ready: 1
2026-04-17T11:53:51.010556374Z Gathering data for ns/openshift-dns...
2026-04-17T11:53:51.529567833Z Gathering data for ns/openshift-image-registry...
2026-04-17T11:53:51.927845982Z Gathering data for ns/openshift-ingress-operator...
2026-04-17T11:53:52.033652480Z Waiting for performance profile collector pods to become ready: 2
2026-04-17T11:53:52.229754391Z Gathering data for ns/openshift-ingress...
2026-04-17T11:53:52.516026119Z Gathering data for ns/openshift-ingress-canary...
2026-04-17T11:53:52.828023075Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T11:53:52.852889899Z Gathering data for ns/openshift-insights...
2026-04-17T11:53:53.153175247Z Waiting for performance profile collector pods to become ready: 3
2026-04-17T11:53:54.277834785Z Waiting for performance profile collector pods to become ready: 4
2026-04-17T11:53:54.616108948Z Gathering data for ns/openshift-jobset-operator...
2026-04-17T11:53:55.349339886Z Gathering data for ns/openshift-config...
2026-04-17T11:53:55.396466670Z Waiting for performance profile collector pods to become ready: 5
2026-04-17T11:53:55.617090836Z Gathering data for ns/openshift-config-managed...
2026-04-17T11:53:55.987031149Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-17T11:53:56.212791095Z Gathering data for ns/openshift-kube-apiserver...
2026-04-17T11:53:56.444534004Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-17T11:53:56.526624342Z Waiting for performance profile collector pods to become ready: 6
2026-04-17T11:53:56.705140587Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-17T11:53:56.994930344Z Gathering data for ns/openshift-kube-scheduler...
2026-04-17T11:53:57.214263019Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-17T11:53:57.470864472Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-17T11:53:57.639320575Z Waiting for performance profile collector pods to become ready: 7
2026-04-17T11:53:57.753756514Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-17T11:53:57.832701809Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T11:53:58.046747273Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-17T11:53:58.470372044Z Gathering data for ns/openshift-multus...
2026-04-17T11:53:58.734918130Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-17T11:53:58.961728014Z Collecting performance related data for node ip-10-0-140-245.ec2.internal
2026-04-17T11:53:58.968646510Z Collecting performance related data for node ip-10-0-134-64.ec2.internal
2026-04-17T11:53:58.976096676Z Collecting performance related data for node ip-10-0-130-210.ec2.internal
2026-04-17T11:53:59.841029933Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-17T11:54:01.226213542Z Gathering data for ns/openshift-host-network...
2026-04-17T11:54:01.548338372Z Gathering data for ns/openshift-network-diagnostics...
2026-04-17T11:54:02.054193491Z Gathering data for ns/openshift-network-node-identity...
2026-04-17T11:54:02.402651130Z Gathering data for ns/openshift-network-operator...
2026-04-17T11:54:02.732673617Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-17T11:54:02.837810912Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T11:54:03.017788155Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-17T11:54:03.445772592Z Gathering data for ns/openshift-apiserver-operator...
2026-04-17T11:54:03.716565571Z Gathering data for ns/openshift-apiserver...
2026-04-17T11:54:04.070290228Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-17T11:54:04.352375487Z Gathering data for ns/openshift-controller-manager...
2026-04-17T11:54:04.584374795Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-17T11:54:05.200490823Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-17T11:54:05.448241029Z Gathering data for ns/openshift-service-ca-operator...
2026-04-17T11:54:05.710219185Z Gathering data for ns/openshift-service-ca...
2026-04-17T11:54:06.019591260Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-17T11:54:06.836291624Z Collecting kubelet logs for node ip-10-0-140-245.ec2.internal
2026-04-17T11:54:06.837948960Z Collecting kubelet logs for node ip-10-0-134-64.ec2.internal
2026-04-17T11:54:06.839699216Z Collecting kubelet logs for node ip-10-0-130-210.ec2.internal
2026-04-17T11:54:06.887420150Z Wrote inspect data to must-gather.
2026-04-17T11:54:06.887476200Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T11:54:06.887476200Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-17T11:54:07.130658499Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-17T11:54:07.135401045Z INFO: Node performance data collection complete.
2026-04-17T11:54:07.259218156Z error: the server doesn't have a resource type "clusters"
2026-04-17T11:54:07.418624160Z Caches written to disk