2026-06-11T12:51:44.487031990Z [disk usage checker] Started
2026-06-11T12:51:44.490925711Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T12:51:44.682722073Z Gathering data for ns/openshift-cluster-version...
2026-06-11T12:51:44.826808745Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-11T12:51:44.902750195Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T12:51:44.974939351Z Gathering data for ns/default...
2026-06-11T12:51:45.229074072Z Gathering data for ns/openshift...
2026-06-11T12:51:45.609540373Z Gathering data for ns/kube-system...
2026-06-11T12:51:46.268456040Z Gathering data for ns/openshift-etcd...
2026-06-11T12:51:46.548859061Z Wrote inspect data to must-gather.
2026-06-11T12:51:46.548877518Z error: inspection completed with the errors occurred while gathering data:
2026-06-11T12:51:46.548877518Z     namespaces "assisted-installer" not found
2026-06-11T12:51:47.030686341Z Waiting on subprocesses to finish execution.
2026-06-11T12:51:47.045349852Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-11T12:51:47.045498861Z INFO: Gathering HAProxy config files
2026-06-11T12:51:47.046148982Z INFO: Collecting host service logs for crio
2026-06-11T12:51:47.046491353Z INFO: Collecting host service logs for kubelet
2026-06-11T12:51:47.046832594Z INFO: Collecting host service logs for rpm-ostreed
2026-06-11T12:51:47.047167903Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-11T12:51:47.047221225Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-11T12:51:47.047548274Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-11T12:51:47.047859153Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-11T12:51:47.047859153Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-11T12:51:47.048164758Z INFO: Collecting host service logs for NetworkManager
2026-06-11T12:51:47.048487287Z INFO: Collecting host service logs for openvswitch
2026-06-11T12:51:47.048802961Z INFO: Collecting host service logs for ovs-configuration
2026-06-11T12:51:47.049232083Z INFO: Collecting host service logs for ovsdb-server
2026-06-11T12:51:47.049557019Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-11T12:51:47.049893888Z INFO: Waiting for worker host service log collection to complete ...
2026-06-11T12:51:47.059352391Z Executing Istio gather script
2026-06-11T12:51:47.158880709Z INFO: Waiting for node performance related collection to complete ...
2026-06-11T12:51:47.354552117Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-11T12:51:47.354640512Z INFO: on-disk MachineConfig config collection complete.
2026-06-11T12:51:47.762066198Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-11T12:51:48.030042446Z No resources found
2026-06-11T12:51:48.090582996Z Wrote inspect data to must-gather.
2026-06-11T12:51:48.107734612Z error: the server doesn't have a resource type "performanceprofile"
2026-06-11T12:51:48.142520124Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-11T12:51:48.270646909Z No resources found in openshift-etcd namespace.
2026-06-11T12:51:48.335908479Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-11T12:51:48.442139146Z INFO: Collecting Insights Archives from insights-operator-896549976-skz6m
2026-06-11T12:51:48.442139146Z insights-runtime-extractor-fdt9r
2026-06-11T12:51:48.442139146Z insights-runtime-extractor-tbg97
2026-06-11T12:51:48.442139146Z insights-runtime-extractor-wphrw
2026-06-11T12:51:48.569164001Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-11T12:51:48.674086394Z INFO: Worker host service log collection to complete.
2026-06-11T12:51:48.760098616Z INFO: "metallb-operator" not detected. Skipping.
2026-06-11T12:51:48.837300922Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-11T12:51:48.888771731Z No resources found
2026-06-11T12:51:49.002730656Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T12:51:49.042588165Z Wrote inspect data to must-gather.
2026-06-11T12:51:49.105091266Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-11T12:51:49.213733309Z ERROR: No running kube-apiserver pods found
2026-06-11T12:51:49.219641389Z Wrote inspect data to must-gather.
2026-06-11T12:51:49.257454615Z Wrote inspect data to must-gather.
2026-06-11T12:51:49.320762292Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-11T12:51:49.320762292Z See 'oc rsync -h' for help and examples
2026-06-11T12:51:49.343125521Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-11T12:51:49.395170099Z Wrote inspect data to must-gather.
2026-06-11T12:51:49.533523031Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T12:51:49.551215361Z Wrote inspect data to must-gather.
2026-06-11T12:51:49.572532905Z Wrote inspect data to must-gather.
2026-06-11T12:51:49.615346614Z tar: Removing leading `/' from member names
2026-06-11T12:51:49.621755958Z INFO: HAProxy config collection complete.
2026-06-11T12:51:49.627771950Z Gathering data for ns/openshift-monitoring...
2026-06-11T12:51:49.640484837Z INFO: 'previous-logs' folder not found on ip-10-0-130-113.ec2.internal, skipping...
2026-06-11T12:51:49.644865419Z error: the server doesn't have a resource type "machineconfigs"
2026-06-11T12:51:49.755290522Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-11T12:51:49.828679660Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-11T12:51:49.889392210Z Wrote inspect data to must-gather.
2026-06-11T12:51:49.922846252Z INFO: Getting rules from prometheus-k8s-0
2026-06-11T12:51:49.954987762Z INFO: 'previous-logs' folder not found on ip-10-0-132-145.ec2.internal, skipping...
2026-06-11T12:51:50.093397266Z Wrote inspect data to must-gather.
2026-06-11T12:51:50.159818932Z Wrote inspect data to must-gather.
2026-06-11T12:51:50.235980352Z INFO: 'previous-logs' folder not found on ip-10-0-133-99.ec2.internal, skipping...
2026-06-11T12:51:50.236025086Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-11T12:51:50.236113337Z INFO: Machine Config Daemon termination log collection complete.
2026-06-11T12:51:50.264870943Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-11T12:51:50.309668392Z INFO: Getting status/config from prometheus-k8s-0
2026-06-11T12:51:50.430995015Z Wrote inspect data to must-gather.
2026-06-11T12:51:50.466913560Z Wrote inspect data to must-gather.
2026-06-11T12:51:50.658529807Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-11T12:51:50.674075144Z Wrote inspect data to must-gather.
2026-06-11T12:51:50.696506427Z INFO: INTERCONNECT MODE
2026-06-11T12:51:50.696552348Z INFO: Gathering ovn-kubernetes DBs
2026-06-11T12:51:50.706377678Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-11T12:51:50.852889525Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:50.863570745Z INFO: Gathering OVN_Northbound from ovnkube-node-bqxvx...
2026-06-11T12:51:50.863792202Z INFO: Gathering OVN_Northbound from ovnkube-node-cd7sp...
2026-06-11T12:51:50.864029865Z INFO: Gathering OVN_Northbound from ovnkube-node-wxfhh...
2026-06-11T12:51:50.864332473Z INFO: Gathering OVN_Southbound from ovnkube-node-bqxvx...
2026-06-11T12:51:50.864823456Z INFO: Gathering OVN_Southbound from ovnkube-node-cd7sp...
2026-06-11T12:51:50.865097044Z INFO: Gathering OVN_Southbound from ovnkube-node-wxfhh...
2026-06-11T12:51:50.923705816Z Wrote inspect data to must-gather.
2026-06-11T12:51:51.025290736Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-11T12:51:51.205244549Z tar: Removing leading `/' from member names
2026-06-11T12:51:51.214166689Z tar: Removing leading `/' from member names
2026-06-11T12:51:51.216715028Z tar: Removing leading `/' from member names
2026-06-11T12:51:51.222992760Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:51.227303030Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-11T12:51:51.233228095Z tar: Removing leading `/' from member names
2026-06-11T12:51:51.236455595Z tar: Removing leading `/' from member names
2026-06-11T12:51:51.237317366Z tar: Removing leading `/' from member names
2026-06-11T12:51:51.271204850Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-11T12:51:51.310296552Z Wrote inspect data to must-gather.
2026-06-11T12:51:51.407166823Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-11T12:51:51.445734429Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:51.554296818Z Wrote inspect data to must-gather.
2026-06-11T12:51:51.690515706Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:51.691710173Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-11T12:51:51.692863846Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-11T12:51:51.735377937Z INFO: Waiting for network log collection to complete ...
2026-06-11T12:51:51.735420012Z INFO: Waiting for ovnk database copies to complete ...
2026-06-11T12:51:51.735914058Z INFO: Copying ovnk databases complete.
2026-06-11T12:51:51.739495858Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-06-11T12:51:51.742746830Z ovnk_database_store/
2026-06-11T12:51:51.742796819Z ovnk_database_store/ovnkube-node-cd7sp_sbdb
2026-06-11T12:51:51.767317674Z ovnk_database_store/ovnkube-node-wxfhh_sbdb
2026-06-11T12:51:51.780024070Z Wrote inspect data to must-gather.
2026-06-11T12:51:51.785457894Z ovnk_database_store/ovnkube-node-wxfhh_nbdb
2026-06-11T12:51:51.792254731Z ovnk_database_store/ovnkube-node-cd7sp_nbdb
2026-06-11T12:51:51.800882071Z ovnk_database_store/ovnkube-node-bqxvx_sbdb
2026-06-11T12:51:51.819225912Z ovnk_database_store/ovnkube-node-bqxvx_nbdb
2026-06-11T12:51:51.829471230Z INFO: Network log collection complete.
2026-06-11T12:51:51.912692206Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:51.933324306Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T12:51:52.081391883Z Wrote inspect data to must-gather.
2026-06-11T12:51:52.085549663Z INFO: Getting status from alertmanager-main-0
2026-06-11T12:51:52.124926816Z Gathering data for ns/openshift-network-console...
2026-06-11T12:51:52.137696790Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:52.140203351Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-11T12:51:52.352964193Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-11T12:51:52.355117293Z INFO: Fallback to identify the container image from release info
2026-06-11T12:51:52.382202827Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:52.553090423Z Gathering data for ns/openshift-console-operator...
2026-06-11T12:51:52.589864134Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:52.592536705Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-11T12:51:52.746121634Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:52.896401641Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-11T12:51:52.932544591Z Gathering data for ns/openshift-console...
2026-06-11T12:51:52.954459087Z Wrote inspect data to /must-gather/istio.
2026-06-11T12:51:53.020361216Z daemonset.apps/perf-node-gather-daemonset created
2026-06-11T12:51:53.149516081Z Waiting for performance profile collector pods to become ready: 1
2026-06-11T12:51:53.419959192Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-11T12:51:53.665150343Z Done executing Istio gather script
2026-06-11T12:51:53.899150038Z Gathering data for ns/openshift-dns-operator...
2026-06-11T12:51:54.148634984Z Gathering data for ns/openshift-dns...
2026-06-11T12:51:54.255085490Z Waiting for performance profile collector pods to become ready: 2
2026-06-11T12:51:54.539244629Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T12:51:54.682420178Z Gathering data for ns/openshift-image-registry...
2026-06-11T12:51:55.084470071Z Gathering data for ns/openshift-ingress-operator...
2026-06-11T12:51:55.384426273Z Waiting for performance profile collector pods to become ready: 3
2026-06-11T12:51:55.394253901Z Gathering data for ns/openshift-ingress...
2026-06-11T12:51:55.683710872Z Gathering data for ns/openshift-ingress-canary...
2026-06-11T12:51:56.031200013Z Gathering data for ns/openshift-insights...
2026-06-11T12:51:56.505144996Z Waiting for performance profile collector pods to become ready: 4
2026-06-11T12:51:57.613961569Z Waiting for performance profile collector pods to become ready: 5
2026-06-11T12:51:57.843425805Z Gathering data for ns/openshift-config...
2026-06-11T12:51:58.079571690Z Gathering data for ns/openshift-config-managed...
2026-06-11T12:51:58.470588141Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-11T12:51:58.718339583Z Gathering data for ns/openshift-kube-apiserver...
2026-06-11T12:51:58.738972576Z Waiting for performance profile collector pods to become ready: 6
2026-06-11T12:51:58.952916196Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-11T12:51:59.205230456Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-11T12:51:59.474944382Z Gathering data for ns/openshift-kube-scheduler...
2026-06-11T12:51:59.544356724Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T12:51:59.727489518Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-11T12:51:59.862313508Z Waiting for performance profile collector pods to become ready: 7
2026-06-11T12:52:00.058203835Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-11T12:52:00.336009771Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-11T12:52:00.628684059Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-11T12:52:00.984325081Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-11T12:52:01.091990012Z Gathering data for ns/openshift-multus...
2026-06-11T12:52:01.222585400Z Collecting performance related data for node ip-10-0-130-113.ec2.internal
2026-06-11T12:52:01.229893306Z Collecting performance related data for node ip-10-0-133-99.ec2.internal
2026-06-11T12:52:01.237883011Z Collecting performance related data for node ip-10-0-132-145.ec2.internal
2026-06-11T12:52:02.601671197Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-11T12:52:03.855420295Z Gathering data for ns/openshift-host-network...
2026-06-11T12:52:04.207341683Z Gathering data for ns/openshift-network-diagnostics...
2026-06-11T12:52:04.549139642Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T12:52:04.684933187Z Gathering data for ns/openshift-network-node-identity...
2026-06-11T12:52:05.057680966Z Gathering data for ns/openshift-network-operator...
2026-06-11T12:52:05.427378756Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-11T12:52:05.662531251Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-11T12:52:06.110568642Z Gathering data for ns/openshift-apiserver-operator...
2026-06-11T12:52:06.378025038Z Gathering data for ns/openshift-apiserver...
2026-06-11T12:52:06.742210757Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-11T12:52:06.975792485Z Gathering data for ns/openshift-controller-manager...
2026-06-11T12:52:07.230676809Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-11T12:52:07.828326627Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-11T12:52:08.106752151Z Gathering data for ns/openshift-service-ca-operator...
2026-06-11T12:52:08.412487983Z Gathering data for ns/openshift-service-ca...
2026-06-11T12:52:08.733573860Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-11T12:52:08.894818508Z Collecting kubelet logs for node ip-10-0-130-113.ec2.internal
2026-06-11T12:52:08.896741708Z Collecting kubelet logs for node ip-10-0-133-99.ec2.internal
2026-06-11T12:52:08.898664486Z Collecting kubelet logs for node ip-10-0-132-145.ec2.internal
2026-06-11T12:52:09.228066412Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-qnjnk namespace
2026-06-11T12:52:09.234355067Z INFO: Node performance data collection complete.
2026-06-11T12:52:09.553879751Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T12:52:09.643307771Z Wrote inspect data to must-gather.
2026-06-11T12:52:09.643353895Z error: inspection completed with the errors occurred while gathering data:
2026-06-11T12:52:09.643353895Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-11T12:52:09.769570924Z error: the server doesn't have a resource type "clusters"
2026-06-11T12:52:09.889501364Z Caches written to disk