2026-06-09T07:42:02.432381046Z [disk usage checker] Started
2026-06-09T07:42:02.436109556Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T07:42:02.631165866Z Gathering data for ns/openshift-cluster-version...
2026-06-09T07:42:02.760086843Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T07:42:02.840225075Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T07:42:02.922983958Z Gathering data for ns/default...
2026-06-09T07:42:03.219834091Z Gathering data for ns/openshift...
2026-06-09T07:42:03.653440081Z Gathering data for ns/kube-system...
2026-06-09T07:42:04.370820379Z Gathering data for ns/openshift-etcd...
2026-06-09T07:42:04.708197539Z Wrote inspect data to must-gather.
2026-06-09T07:42:04.708238843Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T07:42:04.708238843Z     namespaces "assisted-installer" not found
2026-06-09T07:42:05.062629774Z Waiting on subprocesses to finish execution.
2026-06-09T07:42:05.078137050Z Executing Istio gather script
2026-06-09T07:42:05.079761096Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-09T07:42:05.085788218Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-09T07:42:05.085847315Z INFO: Collecting host service logs for crio
2026-06-09T07:42:05.086185925Z INFO: Collecting host service logs for kubelet
2026-06-09T07:42:05.086599125Z INFO: Collecting host service logs for rpm-ostreed
2026-06-09T07:42:05.087062638Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-09T07:42:05.087523073Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-09T07:42:05.087895765Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-09T07:42:05.088294149Z INFO: Collecting host service logs for NetworkManager
2026-06-09T07:42:05.088794276Z INFO: Collecting host service logs for openvswitch
2026-06-09T07:42:05.089171284Z INFO: Collecting host service logs for ovs-configuration
2026-06-09T07:42:05.089331878Z INFO: Gathering HAProxy config files
2026-06-09T07:42:05.089552689Z INFO: Collecting host service logs for ovsdb-server
2026-06-09T07:42:05.089836494Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-09T07:42:05.090229789Z INFO: Waiting for worker host service log collection to complete ...
2026-06-09T07:42:05.092992471Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-09T07:42:05.137036013Z INFO: Waiting for node performance related collection to complete ...
2026-06-09T07:42:05.657758056Z No resources found
2026-06-09T07:42:05.953478909Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-09T07:42:06.033822920Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-09T07:42:06.314258189Z No resources found in openshift-etcd namespace.
2026-06-09T07:42:06.343837503Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-09T07:42:06.343837503Z INFO: on-disk MachineConfig config collection complete.
2026-06-09T07:42:06.408451923Z INFO: Collecting Insights Archives from insights-operator-896549976-jvvkp
2026-06-09T07:42:06.408522511Z insights-runtime-extractor-rf4rq
2026-06-09T07:42:06.408543834Z insights-runtime-extractor-scltq
2026-06-09T07:42:06.408564966Z insights-runtime-extractor-z59vh
2026-06-09T07:42:06.517480584Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-09T07:42:06.574769263Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-09T07:42:06.613753856Z INFO: "metallb-operator" not detected. Skipping.
2026-06-09T07:42:06.701631636Z ERROR: No running kube-apiserver pods found
2026-06-09T07:42:06.748603457Z INFO: Worker host service log collection to complete.
2026-06-09T07:42:06.774388916Z error: the server doesn't have a resource type "performanceprofile"
2026-06-09T07:42:06.800190766Z Wrote inspect data to must-gather.
2026-06-09T07:42:06.863724386Z No resources found
2026-06-09T07:42:06.886114625Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-09T07:42:06.886114625Z See 'oc rsync -h' for help and examples
2026-06-09T07:42:06.891004428Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T07:42:07.050330484Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-09T07:42:07.110204264Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-09T07:42:07.417010038Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-09T07:42:07.463840656Z Wrote inspect data to must-gather.
2026-06-09T07:42:07.472319366Z Wrote inspect data to must-gather.
2026-06-09T07:42:07.504870880Z Wrote inspect data to must-gather.
2026-06-09T07:42:07.514112995Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T07:42:07.530493535Z Wrote inspect data to must-gather.
2026-06-09T07:42:07.533446985Z Wrote inspect data to must-gather.
2026-06-09T07:42:07.555762797Z Wrote inspect data to must-gather.
2026-06-09T07:42:07.658899301Z Gathering data for ns/openshift-monitoring...
2026-06-09T07:42:07.674233207Z tar: Removing leading `/' from member names
2026-06-09T07:42:07.681669616Z INFO: 'previous-logs' folder not found on ip-10-0-131-253.ec2.internal, skipping...
2026-06-09T07:42:07.687391186Z INFO: HAProxy config collection complete.
2026-06-09T07:42:07.795827392Z error: the server doesn't have a resource type "machineconfigs"
2026-06-09T07:42:07.903833038Z Wrote inspect data to must-gather.
2026-06-09T07:42:07.913521232Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-09T07:42:07.981718379Z INFO: 'previous-logs' folder not found on ip-10-0-137-4.ec2.internal, skipping...
2026-06-09T07:42:07.991350887Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-09T07:42:08.051118586Z INFO: Getting rules from prometheus-k8s-0
2026-06-09T07:42:08.183371372Z Wrote inspect data to must-gather.
2026-06-09T07:42:08.241384474Z Wrote inspect data to must-gather.
2026-06-09T07:42:08.257670957Z INFO: 'previous-logs' folder not found on ip-10-0-139-174.ec2.internal, skipping...
2026-06-09T07:42:08.257704137Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-09T07:42:08.257711741Z INFO: Machine Config Daemon termination log collection complete.
2026-06-09T07:42:08.402978475Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-09T07:42:08.437220672Z Wrote inspect data to must-gather.
2026-06-09T07:42:08.490647910Z INFO: Getting status/config from prometheus-k8s-0
2026-06-09T07:42:08.649414935Z Wrote inspect data to must-gather.
2026-06-09T07:42:08.711829565Z INFO: INTERCONNECT MODE
2026-06-09T07:42:08.711868779Z INFO: Gathering ovn-kubernetes DBs
2026-06-09T07:42:08.850962458Z Wrote inspect data to must-gather.
2026-06-09T07:42:08.863909327Z INFO: Gathering OVN_Northbound from ovnkube-node-89q55...
2026-06-09T07:42:08.864128145Z INFO: Gathering OVN_Northbound from ovnkube-node-vvrbk...
2026-06-09T07:42:08.864350397Z INFO: Gathering OVN_Northbound from ovnkube-node-w86jz...
2026-06-09T07:42:08.864660099Z INFO: Gathering OVN_Southbound from ovnkube-node-89q55...
2026-06-09T07:42:08.864968653Z INFO: Gathering OVN_Southbound from ovnkube-node-vvrbk...
2026-06-09T07:42:08.865390100Z INFO: Gathering OVN_Southbound from ovnkube-node-w86jz...
2026-06-09T07:42:08.903193639Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-09T07:42:09.186891983Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-09T07:42:09.266971133Z tar: Removing leading `/' from member names
2026-06-09T07:42:09.267154467Z tar: Removing leading `/' from member names
2026-06-09T07:42:09.281053846Z tar: Removing leading `/' from member names
2026-06-09T07:42:09.282082378Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T07:42:09.290320915Z tar: Removing leading `/' from member names
2026-06-09T07:42:09.293175056Z tar: Removing leading `/' from member names
2026-06-09T07:42:09.298408075Z tar: Removing leading `/' from member names
2026-06-09T07:42:09.305672873Z Wrote inspect data to must-gather.
2026-06-09T07:42:09.327676606Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-09T07:42:09.406104137Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:09.578391682Z Wrote inspect data to must-gather.
2026-06-09T07:42:09.598921415Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-09T07:42:09.664581446Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:09.667357320Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-09T07:42:09.706607069Z INFO: Waiting for network log collection to complete ...
2026-06-09T07:42:09.706737632Z INFO: Waiting for ovnk database copies to complete ...
2026-06-09T07:42:09.707292045Z INFO: Copying ovnk databases complete.
2026-06-09T07:42:09.709459904Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-06-09T07:42:09.712523419Z ovnk_database_store/
2026-06-09T07:42:09.712547503Z ovnk_database_store/ovnkube-node-w86jz_nbdb
2026-06-09T07:42:09.720442623Z ovnk_database_store/ovnkube-node-vvrbk_nbdb
2026-06-09T07:42:09.729380085Z ovnk_database_store/ovnkube-node-89q55_sbdb
2026-06-09T07:42:09.747047513Z ovnk_database_store/ovnkube-node-vvrbk_sbdb
2026-06-09T07:42:09.769827769Z ovnk_database_store/ovnkube-node-89q55_nbdb
2026-06-09T07:42:09.777838916Z ovnk_database_store/ovnkube-node-w86jz_sbdb
2026-06-09T07:42:09.800721113Z INFO: Network log collection complete.
2026-06-09T07:42:09.808423312Z Wrote inspect data to must-gather.
2026-06-09T07:42:09.853885541Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-09T07:42:09.883038085Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:09.957448899Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T07:42:10.115540152Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:10.116977233Z Wrote inspect data to must-gather.
2026-06-09T07:42:10.118958067Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-09T07:42:10.160313496Z Gathering data for ns/openshift-network-console...
2026-06-09T07:42:10.261919889Z INFO: Getting status from alertmanager-main-0
2026-06-09T07:42:10.326985795Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:10.459233777Z Wrote inspect data to must-gather.
2026-06-09T07:42:10.565267633Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:10.568156343Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-09T07:42:10.624977160Z Gathering data for ns/openshift-console-operator...
2026-06-09T07:42:10.737248335Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-09T07:42:10.740669400Z INFO: Fallback to identify the container image from release info
2026-06-09T07:42:10.800801943Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:10.994691413Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:10.996847625Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-09T07:42:11.013976076Z Gathering data for ns/openshift-console...
2026-06-09T07:42:11.167326653Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:11.362924993Z Wrote inspect data to /must-gather/istio.
2026-06-09T07:42:11.416093615Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-09T07:42:11.543954044Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-09T07:42:11.549147357Z daemonset.apps/perf-node-gather-daemonset created
2026-06-09T07:42:11.695443644Z Waiting for performance profile collector pods to become ready: 1
2026-06-09T07:42:12.117375780Z Gathering data for ns/openshift-dns-operator...
2026-06-09T07:42:12.122885223Z Done executing Istio gather script
2026-06-09T07:42:12.408563732Z Gathering data for ns/openshift-dns...
2026-06-09T07:42:12.519402421Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T07:42:12.807855212Z Waiting for performance profile collector pods to become ready: 2
2026-06-09T07:42:12.954278652Z Gathering data for ns/openshift-image-registry...
2026-06-09T07:42:13.380269050Z Gathering data for ns/openshift-ingress-operator...
2026-06-09T07:42:13.698834165Z Gathering data for ns/openshift-ingress...
2026-06-09T07:42:13.920811925Z Waiting for performance profile collector pods to become ready: 3
2026-06-09T07:42:14.017563086Z Gathering data for ns/openshift-ingress-canary...
2026-06-09T07:42:14.370253980Z Gathering data for ns/openshift-insights...
2026-06-09T07:42:15.036707230Z Waiting for performance profile collector pods to become ready: 4
2026-06-09T07:42:16.151785864Z Gathering data for ns/openshift-jobset-operator...
2026-06-09T07:42:16.151785864Z Waiting for performance profile collector pods to become ready: 5
2026-06-09T07:42:16.909222387Z Gathering data for ns/openshift-config...
2026-06-09T07:42:17.159309861Z Gathering data for ns/openshift-config-managed...
2026-06-09T07:42:17.273489055Z Waiting for performance profile collector pods to become ready: 6
2026-06-09T07:42:17.525904578Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T07:42:17.588985398Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-09T07:42:17.829015765Z Gathering data for ns/openshift-kube-apiserver...
2026-06-09T07:42:18.066087810Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-09T07:42:18.328035263Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-09T07:42:18.403540809Z Waiting for performance profile collector pods to become ready: 7
2026-06-09T07:42:18.606004929Z Gathering data for ns/openshift-kube-scheduler...
2026-06-09T07:42:18.895709279Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-09T07:42:19.179367089Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-09T07:42:19.523665257Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-09T07:42:19.538248976Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-09T07:42:19.779760073Z Collecting performance related data for node ip-10-0-137-4.ec2.internal
2026-06-09T07:42:19.789108455Z Collecting performance related data for node ip-10-0-139-174.ec2.internal
2026-06-09T07:42:19.797721353Z Collecting performance related data for node ip-10-0-131-253.ec2.internal
2026-06-09T07:42:19.886677912Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-09T07:42:20.482648510Z Gathering data for ns/openshift-multus...
2026-06-09T07:42:21.890768673Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-09T07:42:22.531474833Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T07:42:23.160341833Z Gathering data for ns/openshift-host-network...
2026-06-09T07:42:23.448456382Z Gathering data for ns/openshift-network-diagnostics...
2026-06-09T07:42:23.970263348Z Gathering data for ns/openshift-network-node-identity...
2026-06-09T07:42:24.411211648Z Gathering data for ns/openshift-network-operator...
2026-06-09T07:42:24.789168088Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-09T07:42:25.054114440Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-09T07:42:25.542650439Z Gathering data for ns/openshift-apiserver-operator...
2026-06-09T07:42:25.842675856Z Gathering data for ns/openshift-apiserver...
2026-06-09T07:42:26.254117908Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-09T07:42:26.492881535Z Gathering data for ns/openshift-controller-manager...
2026-06-09T07:42:26.738889822Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-09T07:42:27.409791394Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-09T07:42:27.538596423Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T07:42:27.680495760Z Gathering data for ns/openshift-service-ca-operator...
2026-06-09T07:42:27.779611189Z Collecting kubelet logs for node ip-10-0-137-4.ec2.internal
2026-06-09T07:42:27.781351724Z Collecting kubelet logs for node ip-10-0-139-174.ec2.internal
2026-06-09T07:42:27.783075139Z Collecting kubelet logs for node ip-10-0-131-253.ec2.internal
2026-06-09T07:42:28.013088018Z Gathering data for ns/openshift-service-ca...
2026-06-09T07:42:28.139844838Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-b7shk namespace
2026-06-09T07:42:28.144609744Z INFO: Node performance data collection complete.
2026-06-09T07:42:28.388414713Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-09T07:42:29.243049834Z Wrote inspect data to must-gather.
2026-06-09T07:42:29.243091736Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T07:42:29.243091736Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-09T07:42:29.367411069Z error: the server doesn't have a resource type "clusters"
2026-06-09T07:42:29.499224489Z Caches written to disk