2026-09-16T13:42:24.997233891Z [disk usage checker] Started
2026-09-16T13:42:25.000427204Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-16T13:42:25.196283242Z Gathering data for ns/openshift-cluster-version...
2026-09-16T13:42:25.330843681Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T13:42:25.425111439Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T13:42:25.509931455Z Gathering data for ns/default...
2026-09-16T13:42:25.822242492Z Gathering data for ns/openshift...
2026-09-16T13:42:26.270211975Z Gathering data for ns/kube-system...
2026-09-16T13:42:26.931358007Z Gathering data for ns/openshift-etcd...
2026-09-16T13:42:27.259259341Z Wrote inspect data to must-gather.
2026-09-16T13:42:27.259291441Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T13:42:27.259291441Z     namespaces "assisted-installer" not found
2026-09-16T13:42:27.687427251Z Waiting on subprocesses to finish execution.
2026-09-16T13:42:27.702107978Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T13:42:27.706912850Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T13:42:27.711620433Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T13:42:27.714981595Z INFO: Collecting host service logs for crio
2026-09-16T13:42:27.716093290Z INFO: Collecting host service logs for kubelet
2026-09-16T13:42:27.716380926Z INFO: Gathering HAProxy config files
2026-09-16T13:42:27.716662702Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T13:42:27.717014107Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T13:42:27.717398658Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T13:42:27.717740679Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T13:42:27.718218531Z INFO: Collecting host service logs for NetworkManager
2026-09-16T13:42:27.719278258Z Executing Istio gather script
2026-09-16T13:42:27.720282911Z INFO: Collecting host service logs for openvswitch
2026-09-16T13:42:27.720629214Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T13:42:27.720991078Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T13:42:27.721444960Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T13:42:27.721855890Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T13:42:27.813863280Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T13:42:28.212206198Z No resources found in openshift-etcd namespace.
2026-09-16T13:42:28.516520448Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T13:42:28.551266176Z Wrote inspect data to must-gather.
2026-09-16T13:42:28.588421917Z INFO: Collecting Insights Archives from insights-operator-5844b856db-28v7h
2026-09-16T13:42:28.588468739Z insights-runtime-extractor-52pvf
2026-09-16T13:42:28.588487827Z insights-runtime-extractor-jmh42
2026-09-16T13:42:28.588506583Z insights-runtime-extractor-np6cj
2026-09-16T13:42:28.677361420Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T13:42:28.697316000Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T13:42:28.697344860Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T13:42:28.902211668Z No resources found
2026-09-16T13:42:29.539381727Z Wrote inspect data to must-gather.
2026-09-16T13:42:29.542905580Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T13:42:29.581674729Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T13:42:29.597700916Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T13:42:29.599551533Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T13:42:29.637807065Z INFO: Worker host service log collection to complete.
2026-09-16T13:42:29.836485593Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T13:42:29.836485593Z See 'oc rsync -h' for help and examples
2026-09-16T13:42:29.892704380Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T13:42:29.929986174Z Wrote inspect data to must-gather.
2026-09-16T13:42:29.964794497Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T13:42:30.006615705Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T13:42:30.007003332Z No resources found
2026-09-16T13:42:30.037652433Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-16T13:42:30.058622528Z ERROR: No running kube-apiserver pods found
2026-09-16T13:42:30.154556649Z Wrote inspect data to must-gather.
2026-09-16T13:42:30.177167017Z Wrote inspect data to must-gather.
2026-09-16T13:42:30.273138007Z Wrote inspect data to must-gather.
2026-09-16T13:42:30.323415182Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T13:42:30.364793501Z Gathering data for ns/openshift-monitoring...
2026-09-16T13:42:30.415567243Z Wrote inspect data to must-gather.
2026-09-16T13:42:30.418495116Z INFO: 'previous-logs' folder not found on ip-10-0-137-26.ec2.internal, skipping...
2026-09-16T13:42:30.497677851Z Wrote inspect data to must-gather.
2026-09-16T13:42:30.617675847Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T13:42:30.636119655Z tar: Removing leading `/' from member names
2026-09-16T13:42:30.640608834Z INFO: HAProxy config collection complete.
2026-09-16T13:42:30.763940650Z INFO: 'previous-logs' folder not found on ip-10-0-139-96.ec2.internal, skipping...
2026-09-16T13:42:30.766927332Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T13:42:30.815670076Z Wrote inspect data to must-gather.
2026-09-16T13:42:30.832635155Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T13:42:31.076643728Z INFO: 'previous-logs' folder not found on ip-10-0-141-139.ec2.internal, skipping...
2026-09-16T13:42:31.076676541Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T13:42:31.076712566Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T13:42:31.104388036Z Wrote inspect data to must-gather.
2026-09-16T13:42:31.108649063Z Wrote inspect data to must-gather.
2026-09-16T13:42:31.275560282Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T13:42:31.282353883Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T13:42:31.336301729Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T13:42:31.453562002Z INFO: INTERCONNECT MODE
2026-09-16T13:42:31.453587218Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T13:42:31.494842878Z Wrote inspect data to must-gather.
2026-09-16T13:42:31.612992532Z INFO: Gathering OVN_Northbound from ovnkube-node-k6ccb...
2026-09-16T13:42:31.613250388Z INFO: Gathering OVN_Northbound from ovnkube-node-wb288...
2026-09-16T13:42:31.613501359Z INFO: Gathering OVN_Northbound from ovnkube-node-wzmzs...
2026-09-16T13:42:31.613772944Z INFO: Gathering OVN_Southbound from ovnkube-node-k6ccb...
2026-09-16T13:42:31.614054781Z INFO: Gathering OVN_Southbound from ovnkube-node-wb288...
2026-09-16T13:42:31.614376571Z INFO: Gathering OVN_Southbound from ovnkube-node-wzmzs...
2026-09-16T13:42:31.775029753Z Wrote inspect data to must-gather.
2026-09-16T13:42:31.828353948Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T13:42:31.915100539Z tar: Removing leading `/' from member names
2026-09-16T13:42:31.953294720Z tar: Removing leading `/' from member names
2026-09-16T13:42:31.973381864Z tar: Removing leading `/' from member names
2026-09-16T13:42:31.996324297Z tar: Removing leading `/' from member names
2026-09-16T13:42:32.038674510Z tar: Removing leading `/' from member names
2026-09-16T13:42:32.065541893Z tar: Removing leading `/' from member names
2026-09-16T13:42:32.120049884Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T13:42:32.148423239Z Wrote inspect data to must-gather.
2026-09-16T13:42:32.161094445Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T13:42:32.412943251Z Wrote inspect data to must-gather.
2026-09-16T13:42:32.417779067Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T13:42:32.554046015Z INFO: Waiting for network log collection to complete ...
2026-09-16T13:42:32.554083984Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T13:42:32.554575906Z INFO: Copying ovnk databases complete.
2026-09-16T13:42:32.556833458Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-09-16T13:42:32.563234996Z ovnk_database_store/
2026-09-16T13:42:32.563234996Z ovnk_database_store/ovnkube-node-wzmzs_sbdb
2026-09-16T13:42:32.592346668Z ovnk_database_store/ovnkube-node-wb288_sbdb
2026-09-16T13:42:32.616949055Z ovnk_database_store/ovnkube-node-k6ccb_nbdb
2026-09-16T13:42:32.625550360Z ovnk_database_store/ovnkube-node-wzmzs_nbdb
2026-09-16T13:42:32.635088146Z ovnk_database_store/ovnkube-node-wb288_nbdb
2026-09-16T13:42:32.645625949Z W0916 13:42:32.645583    1446 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-16T13:42:32.647719592Z ovnk_database_store/ovnkube-node-k6ccb_sbdb
2026-09-16T13:42:32.652794307Z Wrote inspect data to must-gather.
2026-09-16T13:42:32.676372644Z INFO: Network log collection complete.
2026-09-16T13:42:32.676430272Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T13:42:32.706712524Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T13:42:32.773457040Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T13:42:32.922616399Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:32.968539976Z Wrote inspect data to must-gather.
2026-09-16T13:42:32.969231893Z Gathering data for ns/openshift-network-console...
2026-09-16T13:42:33.093663879Z INFO: Getting status from alertmanager-main-0
2026-09-16T13:42:33.164714194Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:33.167847632Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T13:42:33.284340061Z Wrote inspect data to must-gather.
2026-09-16T13:42:33.394627682Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:33.455784637Z Gathering data for ns/openshift-console-operator...
2026-09-16T13:42:33.586516092Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T13:42:33.588227149Z INFO: Fallback to identify the container image from release info
2026-09-16T13:42:33.610201318Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:33.612190181Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T13:42:33.808203864Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:33.857791882Z Gathering data for ns/openshift-console...
2026-09-16T13:42:34.020608835Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:34.023434777Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T13:42:34.248519398Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:34.323595103Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T13:42:34.392480486Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T13:42:34.460481147Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:34.463729095Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-16T13:42:34.472444311Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T13:42:34.613370472Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T13:42:34.646318266Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:34.863228767Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:34.867070680Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-16T13:42:34.958176816Z Gathering data for ns/openshift-dns-operator...
2026-09-16T13:42:35.073689783Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:35.074772471Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-16T13:42:35.281932199Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:35.284290902Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T13:42:35.303236898Z Gathering data for ns/openshift-dns...
2026-09-16T13:42:35.443610906Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:35.633979110Z Wrote inspect data to /must-gather/istio.
2026-09-16T13:42:35.755531301Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T13:42:35.904705736Z Gathering data for ns/openshift-image-registry...
2026-09-16T13:42:36.348555168Z Done executing Istio gather script
2026-09-16T13:42:36.385890678Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T13:42:36.724910682Z Gathering data for ns/openshift-ingress...
2026-09-16T13:42:36.881110899Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T13:42:37.045398394Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T13:42:37.386976861Z Gathering data for ns/openshift-insights...
2026-09-16T13:42:38.005909320Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T13:42:39.121761825Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T13:42:39.760696286Z Gathering data for ns/kserve...
2026-09-16T13:42:40.079289670Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-16T13:42:40.233995541Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T13:42:41.366024459Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T13:42:41.406744996Z Gathering data for ns/openshift-config...
2026-09-16T13:42:41.665402680Z Gathering data for ns/openshift-config-managed...
2026-09-16T13:42:42.063579729Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T13:42:42.306072763Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T13:42:42.484613350Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T13:42:42.579507306Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T13:42:42.760195956Z Collecting performance related data for node ip-10-0-141-139.ec2.internal
2026-09-16T13:42:42.766892769Z Collecting performance related data for node ip-10-0-139-96.ec2.internal
2026-09-16T13:42:42.774152863Z Collecting performance related data for node ip-10-0-137-26.ec2.internal
2026-09-16T13:42:42.905825628Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T13:42:43.250550456Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T13:42:43.567606623Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T13:42:43.938328111Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T13:42:44.374544445Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T13:42:44.676899455Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T13:42:45.086610428Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-16T13:42:45.203520233Z Gathering data for ns/openshift-multus...
2026-09-16T13:42:46.519798850Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T13:42:47.817316965Z Gathering data for ns/openshift-host-network...
2026-09-16T13:42:48.128077443Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T13:42:48.653640574Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T13:42:49.047154702Z Gathering data for ns/openshift-network-operator...
2026-09-16T13:42:49.396862239Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T13:42:49.648155343Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T13:42:50.093576120Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-09-16T13:42:50.160739736Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T13:42:50.445324592Z Gathering data for ns/openshift-apiserver...
2026-09-16T13:42:50.809216315Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T13:42:50.864583533Z Collecting kubelet logs for node ip-10-0-141-139.ec2.internal
2026-09-16T13:42:50.867248454Z Collecting kubelet logs for node ip-10-0-139-96.ec2.internal
2026-09-16T13:42:50.869509582Z Collecting kubelet logs for node ip-10-0-137-26.ec2.internal
2026-09-16T13:42:51.128168229Z Gathering data for ns/openshift-controller-manager...
2026-09-16T13:42:51.267582510Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-svrn2 namespace
2026-09-16T13:42:51.273073645Z INFO: Node performance data collection complete.
2026-09-16T13:42:51.463827956Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T13:42:52.080394725Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T13:42:52.353635303Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T13:42:52.648279684Z Gathering data for ns/openshift-service-ca...
2026-09-16T13:42:52.986748368Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T13:42:53.823234268Z Wrote inspect data to must-gather.
2026-09-16T13:42:53.823283413Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T13:42:53.823283413Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T13:42:53.951047148Z error: the server doesn't have a resource type "clusters"
2026-09-16T13:42:54.135300705Z Caches written to disk