2026-07-06T19:16:22.818634864Z [disk usage checker] Started
2026-07-06T19:16:22.822465691Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-06T19:16:23.036306840Z Gathering data for ns/openshift-cluster-version...
2026-07-06T19:16:23.215376115Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T19:16:23.331303394Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T19:16:23.443058836Z Gathering data for ns/default...
2026-07-06T19:16:23.813910652Z Gathering data for ns/openshift...
2026-07-06T19:16:24.347586562Z Gathering data for ns/kube-system...
2026-07-06T19:16:25.144126679Z Gathering data for ns/openshift-etcd...
2026-07-06T19:16:25.529709225Z Wrote inspect data to must-gather.
2026-07-06T19:16:25.529745962Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T19:16:25.529745962Z     namespaces "assisted-installer" not found
2026-07-06T19:16:25.625604664Z Waiting on subprocesses to finish execution.
2026-07-06T19:16:25.633908383Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-06T19:16:25.637942697Z INFO: Gathering HAProxy config files
2026-07-06T19:16:25.638658821Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-06T19:16:25.639889213Z Executing Istio gather script
2026-07-06T19:16:25.642092694Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-06T19:16:25.646517895Z INFO: Collecting host service logs for crio
2026-07-06T19:16:25.646832390Z INFO: Collecting host service logs for kubelet
2026-07-06T19:16:25.647124722Z INFO: Collecting host service logs for rpm-ostreed
2026-07-06T19:16:25.647428474Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-06T19:16:25.647799407Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-06T19:16:25.648094774Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-06T19:16:25.648386062Z INFO: Collecting host service logs for NetworkManager
2026-07-06T19:16:25.648705490Z INFO: Collecting host service logs for openvswitch
2026-07-06T19:16:25.649007883Z INFO: Collecting host service logs for ovs-configuration
2026-07-06T19:16:25.649323334Z INFO: Collecting host service logs for ovsdb-server
2026-07-06T19:16:25.649605550Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-06T19:16:25.650057658Z INFO: Waiting for worker host service log collection to complete ...
2026-07-06T19:16:25.663132767Z INFO: Waiting for node performance related collection to complete ...
2026-07-06T19:16:26.222195082Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-06T19:16:26.448309773Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-06T19:16:26.473546446Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-06T19:16:26.473645224Z INFO: on-disk MachineConfig config collection complete.
2026-07-06T19:16:26.798718889Z error: the server doesn't have a resource type "performanceprofile"
2026-07-06T19:16:26.903585077Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-06T19:16:26.987438058Z INFO: "metallb-operator" not detected. Skipping.
2026-07-06T19:16:27.066627151Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-2scgt
2026-07-06T19:16:27.066712241Z insights-runtime-extractor-2ssph
2026-07-06T19:16:27.066746754Z insights-runtime-extractor-fw88z
2026-07-06T19:16:27.066790431Z insights-runtime-extractor-zsddx
2026-07-06T19:16:27.091764007Z No resources found in openshift-etcd namespace.
2026-07-06T19:16:27.236489506Z No resources found
2026-07-06T19:16:27.238667627Z INFO: Worker host service log collection to complete.
2026-07-06T19:16:27.562750414Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-06T19:16:27.667236200Z Wrote inspect data to must-gather.
2026-07-06T19:16:27.796555577Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-06T19:16:27.837499460Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-06T19:16:27.850584231Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T19:16:27.861381898Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-06T19:16:27.902842984Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-06T19:16:27.907384301Z Wrote inspect data to must-gather.
2026-07-06T19:16:27.950189160Z No resources found
2026-07-06T19:16:28.084704141Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-06T19:16:28.084704141Z See 'oc rsync -h' for help and examples
2026-07-06T19:16:28.110764524Z ERROR: No running kube-apiserver pods found
2026-07-06T19:16:28.294194358Z Wrote inspect data to must-gather.
2026-07-06T19:16:28.311916128Z Wrote inspect data to must-gather.
2026-07-06T19:16:28.315099825Z Wrote inspect data to must-gather.
2026-07-06T19:16:28.320034873Z Wrote inspect data to must-gather.
2026-07-06T19:16:28.375524449Z tar: Removing leading `/' from member names
2026-07-06T19:16:28.381178702Z Wrote inspect data to must-gather.
2026-07-06T19:16:28.382708793Z INFO: HAProxy config collection complete.
2026-07-06T19:16:28.496406180Z INFO: 'previous-logs' folder not found on ip-10-0-131-229.ec2.internal, skipping...
2026-07-06T19:16:28.569521321Z error: the server doesn't have a resource type "machineconfigs"
2026-07-06T19:16:28.668802385Z INFO: Getting rules from prometheus-k8s-0
2026-07-06T19:16:28.676603517Z Wrote inspect data to must-gather.
2026-07-06T19:16:28.770269233Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-06T19:16:28.889713369Z INFO: 'previous-logs' folder not found on ip-10-0-133-206.ec2.internal, skipping...
2026-07-06T19:16:28.927274191Z Gathering data for ns/openshift-monitoring...
2026-07-06T19:16:29.017440978Z Wrote inspect data to must-gather.
2026-07-06T19:16:29.087038547Z Wrote inspect data to must-gather.
2026-07-06T19:16:29.206939308Z INFO: Getting status/config from prometheus-k8s-0
2026-07-06T19:16:29.222176436Z INFO: 'previous-logs' folder not found on ip-10-0-137-248.ec2.internal, skipping...
2026-07-06T19:16:29.222212963Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-06T19:16:29.222252873Z INFO: Machine Config Daemon termination log collection complete.
2026-07-06T19:16:29.270642778Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-06T19:16:29.298117672Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-06T19:16:29.359487529Z Wrote inspect data to must-gather.
2026-07-06T19:16:29.530829235Z Wrote inspect data to must-gather.
2026-07-06T19:16:29.672214537Z INFO: INTERCONNECT MODE
2026-07-06T19:16:29.672249787Z INFO: Gathering ovn-kubernetes DBs
2026-07-06T19:16:29.692173192Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-06T19:16:29.776812265Z Wrote inspect data to must-gather.
2026-07-06T19:16:29.842290101Z INFO: Gathering OVN_Northbound from ovnkube-node-jjzrs...
2026-07-06T19:16:29.842485136Z INFO: Gathering OVN_Northbound from ovnkube-node-krs87...
2026-07-06T19:16:29.842711093Z INFO: Gathering OVN_Northbound from ovnkube-node-nchfw...
2026-07-06T19:16:29.843020785Z INFO: Gathering OVN_Southbound from ovnkube-node-jjzrs...
2026-07-06T19:16:29.843241837Z INFO: Gathering OVN_Southbound from ovnkube-node-krs87...
2026-07-06T19:16:29.843517507Z INFO: Gathering OVN_Southbound from ovnkube-node-nchfw...
2026-07-06T19:16:29.987567818Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-06T19:16:30.126198680Z Wrote inspect data to must-gather.
2026-07-06T19:16:30.137600440Z tar: Removing leading `/' from member names
2026-07-06T19:16:30.145286862Z tar: Removing leading `/' from member names
2026-07-06T19:16:30.149587452Z tar: Removing leading `/' from member names
2026-07-06T19:16:30.150600388Z tar: Removing leading `/' from member names
2026-07-06T19:16:30.194206208Z tar: Removing leading `/' from member names
2026-07-06T19:16:30.222028436Z tar: Removing leading `/' from member names
2026-07-06T19:16:30.291338235Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-06T19:16:30.322066366Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-06T19:16:30.365025452Z Wrote inspect data to must-gather.
2026-07-06T19:16:30.511128737Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:30.600871546Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-06T19:16:30.625919160Z Wrote inspect data to must-gather.
2026-07-06T19:16:30.670372201Z INFO: Waiting for network log collection to complete ...
2026-07-06T19:16:30.670407333Z INFO: Waiting for ovnk database copies to complete ...
2026-07-06T19:16:30.671085207Z INFO: Copying ovnk databases complete.
2026-07-06T19:16:30.673432515Z 3.0M	must-gather/network_logs/ovnk_database_store
2026-07-06T19:16:30.676495674Z ovnk_database_store/
2026-07-06T19:16:30.676544487Z ovnk_database_store/ovnkube-node-krs87_sbdb
2026-07-06T19:16:30.696202494Z ovnk_database_store/ovnkube-node-nchfw_nbdb
2026-07-06T19:16:30.703645702Z ovnk_database_store/ovnkube-node-jjzrs_nbdb
2026-07-06T19:16:30.711196285Z ovnk_database_store/ovnkube-node-nchfw_sbdb
2026-07-06T19:16:30.728572815Z ovnk_database_store/ovnkube-node-krs87_nbdb
2026-07-06T19:16:30.735633484Z ovnk_database_store/ovnkube-node-jjzrs_sbdb
2026-07-06T19:16:30.743644249Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:30.747469617Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-06T19:16:30.757866961Z INFO: Network log collection complete.
2026-07-06T19:16:30.881494729Z Wrote inspect data to must-gather.
2026-07-06T19:16:30.959376799Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:31.036577234Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T19:16:31.051477439Z INFO: Getting status from alertmanager-main-0
2026-07-06T19:16:31.110243029Z Wrote inspect data to must-gather.
2026-07-06T19:16:31.195323789Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:31.198620020Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-06T19:16:31.400827053Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:31.417183282Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-06T19:16:31.419849822Z INFO: Fallback to identify the container image from release info
2026-07-06T19:16:31.599944286Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:31.603960219Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-06T19:16:31.664533710Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T19:16:31.824650841Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:31.853917216Z Gathering data for ns/openshift-network-console...
2026-07-06T19:16:32.028610579Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:32.031341482Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io
2026-07-06T19:16:32.113434441Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-06T19:16:32.232160406Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:32.257874562Z daemonset.apps/perf-node-gather-daemonset created
2026-07-06T19:16:32.406744101Z Gathering data for ns/openshift-console-operator...
2026-07-06T19:16:32.408457225Z Waiting for performance profile collector pods to become ready: 1
2026-07-06T19:16:32.455078847Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:32.458333662Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io
2026-07-06T19:16:32.632090903Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:32.870453169Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:32.874169778Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io
2026-07-06T19:16:32.909145722Z Gathering data for ns/openshift-console...
2026-07-06T19:16:32.914620723Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-06T19:16:33.055572663Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:33.241203556Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:33.243475790Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-06T19:16:33.423819528Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:33.543876263Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-06T19:16:33.560542084Z Waiting for performance profile collector pods to become ready: 2
2026-07-06T19:16:33.643430039Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:33.647016789Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-06T19:16:33.804001861Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:34.002976852Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:34.005268457Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-06T19:16:34.191834267Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:34.230885086Z Gathering data for ns/openshift-dns-operator...
2026-07-06T19:16:34.395719174Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:16:34.653995702Z Gathering data for ns/openshift-dns...
2026-07-06T19:16:34.698492691Z Waiting for performance profile collector pods to become ready: 3
2026-07-06T19:16:35.095756515Z Done executing Istio gather script
2026-07-06T19:16:35.439202948Z Gathering data for ns/openshift-image-registry...
2026-07-06T19:16:35.825356517Z Waiting for performance profile collector pods to become ready: 4
2026-07-06T19:16:36.026301324Z Gathering data for ns/openshift-ingress-operator...
2026-07-06T19:16:36.488877308Z Gathering data for ns/openshift-ingress...
2026-07-06T19:16:36.941900453Z Gathering data for ns/openshift-ingress-canary...
2026-07-06T19:16:36.952415985Z Waiting for performance profile collector pods to become ready: 5
2026-07-06T19:16:37.406714375Z Gathering data for ns/openshift-insights...
2026-07-06T19:16:37.919103729Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-06T19:16:38.078666098Z Waiting for performance profile collector pods to become ready: 6
2026-07-06T19:16:39.201782138Z Waiting for performance profile collector pods to become ready: 7
2026-07-06T19:16:39.816919550Z Gathering data for ns/kserve...
2026-07-06T19:16:40.328015621Z Waiting for performance profile collector pods to become ready: 8
2026-07-06T19:16:41.345894368Z Gathering data for ns/openshift-config...
2026-07-06T19:16:41.459156657Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-06T19:16:41.707277540Z Gathering data for ns/openshift-config-managed...
2026-07-06T19:16:41.728124208Z Collecting performance related data for node ip-10-0-133-206.ec2.internal
2026-07-06T19:16:41.736981056Z Collecting performance related data for node ip-10-0-137-248.ec2.internal
2026-07-06T19:16:41.746881341Z Collecting performance related data for node ip-10-0-131-229.ec2.internal
2026-07-06T19:16:42.355744230Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-06T19:16:42.772960437Z Gathering data for ns/openshift-kube-apiserver...
2026-07-06T19:16:42.923963177Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-06T19:16:43.187003434Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-06T19:16:43.585250164Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-06T19:16:43.962272946Z Gathering data for ns/openshift-kube-scheduler...
2026-07-06T19:16:44.366420614Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-06T19:16:44.829362069Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-06T19:16:45.232363005Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-06T19:16:45.616278521Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-06T19:16:46.201133426Z Gathering data for ns/openshift-multus...
2026-07-06T19:16:47.928579611Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-06T19:16:48.096375518Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-06T19:16:49.616078227Z Gathering data for ns/openshift-host-network...
2026-07-06T19:16:50.076414342Z Gathering data for ns/openshift-network-diagnostics...
2026-07-06T19:16:50.187241016Z Collecting kubelet logs for node ip-10-0-133-206.ec2.internal
2026-07-06T19:16:50.189032795Z Collecting kubelet logs for node ip-10-0-137-248.ec2.internal
2026-07-06T19:16:50.190713342Z Collecting kubelet logs for node ip-10-0-131-229.ec2.internal
2026-07-06T19:16:50.558369907Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-qck8h namespace
2026-07-06T19:16:50.565034290Z INFO: Node performance data collection complete.
2026-07-06T19:16:50.814109002Z Gathering data for ns/openshift-network-node-identity...
2026-07-06T19:16:51.401745680Z Gathering data for ns/openshift-network-operator...
2026-07-06T19:16:51.920062371Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-06T19:16:52.261892571Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-06T19:16:52.903367883Z Gathering data for ns/openshift-apiserver-operator...
2026-07-06T19:16:52.934876647Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-06T19:16:53.286989690Z Gathering data for ns/openshift-apiserver...
2026-07-06T19:16:53.819440882Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-06T19:16:54.223167764Z Gathering data for ns/openshift-controller-manager...
2026-07-06T19:16:54.621180834Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-06T19:16:55.328952884Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-06T19:16:55.715811653Z Gathering data for ns/openshift-service-ca-operator...
2026-07-06T19:16:56.124126925Z Gathering data for ns/openshift-service-ca...
2026-07-06T19:16:56.636330433Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-06T19:16:57.633480782Z Wrote inspect data to must-gather.
2026-07-06T19:16:57.633525079Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T19:16:57.633525079Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-06T19:16:57.762399066Z error: the server doesn't have a resource type "clusters"
2026-07-06T19:16:57.901881825Z Caches written to disk