2026-04-23T18:20:52.310405967Z [disk usage checker] Started
2026-04-23T18:20:52.367966528Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-23T18:20:52.534709411Z Gathering data for ns/openshift-cluster-version...
2026-04-23T18:20:52.679638445Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-23T18:20:52.779971853Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T18:20:52.872239569Z Gathering data for ns/default...
2026-04-23T18:20:53.208805832Z Gathering data for ns/openshift...
2026-04-23T18:20:53.691315832Z Gathering data for ns/kube-system...
2026-04-23T18:20:54.537426943Z Gathering data for ns/openshift-etcd...
2026-04-23T18:20:54.878932417Z Wrote inspect data to must-gather.
2026-04-23T18:20:54.878967621Z error: inspection completed with the errors occurred while gathering data:
2026-04-23T18:20:54.878967621Z     namespaces "assisted-installer" not found
2026-04-23T18:20:54.934017767Z Waiting on subprocesses to finish execution.
2026-04-23T18:20:54.945743857Z INFO: Gathering HAProxy config files
2026-04-23T18:20:54.946932120Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-23T18:20:54.948066935Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-23T18:20:54.956931931Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-23T18:20:54.973168241Z INFO: Collecting host service logs for crio
2026-04-23T18:20:54.973616529Z INFO: Collecting host service logs for kubelet
2026-04-23T18:20:54.973993836Z INFO: Collecting host service logs for rpm-ostreed
2026-04-23T18:20:54.974327501Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-23T18:20:54.974847551Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-23T18:20:54.975199146Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-23T18:20:54.975532035Z INFO: Collecting host service logs for NetworkManager
2026-04-23T18:20:54.975888408Z INFO: Collecting host service logs for openvswitch
2026-04-23T18:20:54.976231120Z INFO: Collecting host service logs for ovs-configuration
2026-04-23T18:20:54.976584945Z INFO: Collecting host service logs for ovsdb-server
2026-04-23T18:20:54.977011877Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-23T18:20:54.977500929Z INFO: Waiting for worker host service log collection to complete ...
2026-04-23T18:20:55.174089434Z INFO: Waiting for node performance related collection to complete ...
2026-04-23T18:20:55.439950076Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-23T18:20:55.519534663Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-dsmq2
2026-04-23T18:20:55.519591372Z insights-runtime-extractor-4jc8x
2026-04-23T18:20:55.519611654Z insights-runtime-extractor-89vtf
2026-04-23T18:20:55.519632811Z insights-runtime-extractor-rwkcd
2026-04-23T18:20:55.886673843Z Wrote inspect data to must-gather.
2026-04-23T18:20:55.941619982Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-23T18:20:55.977417141Z No resources found
2026-04-23T18:20:56.007225974Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-23T18:20:56.007340456Z INFO: on-disk MachineConfig config collection complete.
2026-04-23T18:20:56.106107223Z No resources found in openshift-etcd namespace.
2026-04-23T18:20:56.312384838Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-23T18:20:56.346818719Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-23T18:20:56.346818719Z See 'oc rsync -h' for help and examples
2026-04-23T18:20:56.393844593Z error: the server doesn't have a resource type "performanceprofile"
2026-04-23T18:20:56.419799844Z INFO: Worker host service log collection to complete.
2026-04-23T18:20:56.429310960Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-23T18:20:56.555638930Z ERROR: No running kube-apiserver pods found
2026-04-23T18:20:56.571389388Z INFO: "metallb-operator" not detected. Skipping.
2026-04-23T18:20:56.927284629Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-23T18:20:56.989437484Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.002688700Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.056438915Z INFO: 'previous-logs' folder not found on ip-10-0-131-107.ec2.internal, skipping...
2026-04-23T18:20:57.097745754Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T18:20:57.135132765Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.283292340Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-23T18:20:57.286860251Z No resources found
2026-04-23T18:20:57.287210829Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.311538252Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.315590753Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-23T18:20:57.346625460Z Gathering data for ns/openshift-monitoring...
2026-04-23T18:20:57.395405466Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-23T18:20:57.395752085Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.538474991Z error: the server doesn't have a resource type "machineconfigs"
2026-04-23T18:20:57.567679969Z tar: Removing leading `/' from member names
2026-04-23T18:20:57.575637600Z INFO: HAProxy config collection complete.
2026-04-23T18:20:57.591896810Z INFO: 'previous-logs' folder not found on ip-10-0-138-17.ec2.internal, skipping...
2026-04-23T18:20:57.685399191Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.696269784Z INFO: Getting rules from prometheus-k8s-0
2026-04-23T18:20:57.796877074Z Wrote inspect data to must-gather.
2026-04-23T18:20:57.903899254Z INFO: 'previous-logs' folder not found on ip-10-0-139-215.ec2.internal, skipping...
2026-04-23T18:20:57.903944051Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-23T18:20:57.903944051Z INFO: Machine Config Daemon termination log collection complete.
2026-04-23T18:20:57.936688345Z Wrote inspect data to must-gather.
2026-04-23T18:20:58.037096944Z Wrote inspect data to must-gather.
2026-04-23T18:20:58.123079309Z INFO: Getting status/config from prometheus-k8s-0
2026-04-23T18:20:58.193917582Z Wrote inspect data to must-gather.
2026-04-23T18:20:58.205738722Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-23T18:20:58.418161971Z Wrote inspect data to must-gather.
2026-04-23T18:20:58.508818142Z INFO: INTERCONNECT MODE
2026-04-23T18:20:58.508857114Z INFO: Gathering ovn-kubernetes DBs
2026-04-23T18:20:58.542529242Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-23T18:20:58.652475620Z Wrote inspect data to must-gather.
2026-04-23T18:20:58.699262063Z INFO: Gathering OVN_Northbound from ovnkube-node-246wr...
2026-04-23T18:20:58.699461902Z INFO: Gathering OVN_Northbound from ovnkube-node-cqkbt...
2026-04-23T18:20:58.699646914Z INFO: Gathering OVN_Northbound from ovnkube-node-zxwk2...
2026-04-23T18:20:58.699989426Z INFO: Gathering OVN_Southbound from ovnkube-node-246wr...
2026-04-23T18:20:58.700234433Z INFO: Gathering OVN_Southbound from ovnkube-node-cqkbt...
2026-04-23T18:20:58.700698772Z INFO: Gathering OVN_Southbound from ovnkube-node-zxwk2...
2026-04-23T18:20:58.886947545Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-23T18:20:58.979842653Z Wrote inspect data to must-gather.
2026-04-23T18:20:58.995811049Z tar: Removing leading `/' from member names
2026-04-23T18:20:59.003490495Z tar: Removing leading `/' from member names
2026-04-23T18:20:59.016343031Z tar: Removing leading `/' from member names
2026-04-23T18:20:59.017220020Z tar: Removing leading `/' from member names
2026-04-23T18:20:59.017969139Z tar: Removing leading `/' from member names
2026-04-23T18:20:59.030339639Z tar: Removing leading `/' from member names
2026-04-23T18:20:59.157103141Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-23T18:20:59.205717118Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-23T18:20:59.224327356Z Wrote inspect data to must-gather.
2026-04-23T18:20:59.477628362Z Wrote inspect data to must-gather.
2026-04-23T18:20:59.492726431Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-23T18:20:59.511443753Z INFO: Waiting for network log collection to complete ...
2026-04-23T18:20:59.511686662Z INFO: Waiting for ovnk database copies to complete ...
2026-04-23T18:20:59.512222191Z INFO: Copying ovnk databases complete.
2026-04-23T18:20:59.514416638Z 3.2M	must-gather/network_logs/ovnk_database_store
2026-04-23T18:20:59.517119400Z ovnk_database_store/
2026-04-23T18:20:59.517119400Z ovnk_database_store/ovnkube-node-cqkbt_sbdb
2026-04-23T18:20:59.538624119Z ovnk_database_store/ovnkube-node-cqkbt_nbdb
2026-04-23T18:20:59.546250671Z ovnk_database_store/ovnkube-node-246wr_nbdb
2026-04-23T18:20:59.554715861Z ovnk_database_store/ovnkube-node-zxwk2_nbdb
2026-04-23T18:20:59.564653696Z ovnk_database_store/ovnkube-node-zxwk2_sbdb
2026-04-23T18:20:59.587794934Z ovnk_database_store/ovnkube-node-246wr_sbdb
2026-04-23T18:20:59.613237009Z INFO: Network log collection complete.
2026-04-23T18:20:59.722750459Z W0423 18:20:59.722694    1378 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-23T18:20:59.728533314Z Wrote inspect data to must-gather.
2026-04-23T18:20:59.791123786Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T18:20:59.935759055Z INFO: Getting status from alertmanager-main-0
2026-04-23T18:20:59.978137634Z Gathering data for ns/openshift-network-console...
2026-04-23T18:21:00.044073297Z Wrote inspect data to must-gather.
2026-04-23T18:21:00.326937777Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-23T18:21:00.329312535Z INFO: Fallback to identify the container image from release info
2026-04-23T18:21:00.454307545Z Gathering data for ns/openshift-console-operator...
2026-04-23T18:21:00.838652683Z Gathering data for ns/openshift-console...
2026-04-23T18:21:00.904364878Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-23T18:21:01.049803457Z daemonset.apps/perf-node-gather-daemonset created
2026-04-23T18:21:01.193095438Z Waiting for performance profile collector pods to become ready: 1
2026-04-23T18:21:01.327637705Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-23T18:21:01.719390941Z Gathering data for ns/openshift-dns-operator...
2026-04-23T18:21:02.060935917Z Gathering data for ns/openshift-dns...
2026-04-23T18:21:02.318145430Z Waiting for performance profile collector pods to become ready: 2
2026-04-23T18:21:02.400643896Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-23T18:21:02.714404908Z Gathering data for ns/openshift-image-registry...
2026-04-23T18:21:03.222739634Z Gathering data for ns/openshift-ingress-operator...
2026-04-23T18:21:03.449563263Z Waiting for performance profile collector pods to become ready: 3
2026-04-23T18:21:03.606738603Z Gathering data for ns/openshift-ingress...
2026-04-23T18:21:03.958458306Z Gathering data for ns/openshift-ingress-canary...
2026-04-23T18:21:04.374520105Z Gathering data for ns/openshift-insights...
2026-04-23T18:21:04.580934704Z Waiting for performance profile collector pods to become ready: 4
2026-04-23T18:21:05.715810880Z Waiting for performance profile collector pods to become ready: 5
2026-04-23T18:21:06.619999344Z Gathering data for ns/kserve...
2026-04-23T18:21:06.839848510Z Waiting for performance profile collector pods to become ready: 6
2026-04-23T18:21:07.406402192Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-23T18:21:07.971665264Z Waiting for performance profile collector pods to become ready: 7
2026-04-23T18:21:08.099291353Z Gathering data for ns/openshift-config...
2026-04-23T18:21:08.435291819Z Gathering data for ns/openshift-config-managed...
2026-04-23T18:21:08.897544297Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-23T18:21:09.100843994Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-23T18:21:09.204878016Z Gathering data for ns/openshift-kube-apiserver...
2026-04-23T18:21:09.440593445Z Collecting performance related data for node ip-10-0-139-215.ec2.internal
2026-04-23T18:21:09.449936135Z Collecting performance related data for node ip-10-0-138-17.ec2.internal
2026-04-23T18:21:09.462346099Z Collecting performance related data for node ip-10-0-131-107.ec2.internal
2026-04-23T18:21:09.577075803Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-23T18:21:09.977136430Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-23T18:21:10.367511612Z Gathering data for ns/openshift-kube-scheduler...
2026-04-23T18:21:10.733250252Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-23T18:21:11.136702620Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-23T18:21:11.514561783Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-23T18:21:11.865302426Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-23T18:21:12.411874315Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-23T18:21:12.467089040Z Gathering data for ns/openshift-multus...
2026-04-23T18:21:14.006919887Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-23T18:21:15.812950230Z Gathering data for ns/openshift-host-network...
2026-04-23T18:21:16.162674512Z Gathering data for ns/openshift-network-diagnostics...
2026-04-23T18:21:16.822537937Z Gathering data for ns/openshift-network-node-identity...
2026-04-23T18:21:17.332282801Z Gathering data for ns/openshift-network-operator...
2026-04-23T18:21:17.417681642Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-23T18:21:17.768231553Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-23T18:21:17.902757080Z Collecting kubelet logs for node ip-10-0-139-215.ec2.internal
2026-04-23T18:21:17.904542771Z Collecting kubelet logs for node ip-10-0-138-17.ec2.internal
2026-04-23T18:21:17.906358597Z Collecting kubelet logs for node ip-10-0-131-107.ec2.internal
2026-04-23T18:21:18.108190284Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-23T18:21:18.319840876Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-23T18:21:18.324973084Z INFO: Node performance data collection complete.
2026-04-23T18:21:18.657115167Z Gathering data for ns/openshift-apiserver-operator...
2026-04-23T18:21:18.965457840Z Gathering data for ns/openshift-apiserver...
2026-04-23T18:21:19.392033660Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-23T18:21:19.676003760Z Gathering data for ns/openshift-controller-manager...
2026-04-23T18:21:19.966429034Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-23T18:21:20.688622355Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-23T18:21:21.009390850Z Gathering data for ns/openshift-service-ca-operator...
2026-04-23T18:21:21.356958212Z Gathering data for ns/openshift-service-ca...
2026-04-23T18:21:21.734974027Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-23T18:21:22.422662856Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-23T18:21:22.663105775Z Wrote inspect data to must-gather.
2026-04-23T18:21:22.663152613Z error: inspection completed with the errors occurred while gathering data:
2026-04-23T18:21:22.663152613Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-23T18:21:22.806478516Z error: the server doesn't have a resource type "clusters"
2026-04-23T18:21:23.004481835Z Caches written to disk