2026-09-11T02:54:40.429729541Z [disk usage checker] Started
2026-09-11T02:54:40.433439516Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-11T02:54:40.633688763Z Gathering data for ns/openshift-cluster-version...
2026-09-11T02:54:40.818422344Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T02:54:40.931692533Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T02:54:41.053618985Z Gathering data for ns/default...
2026-09-11T02:54:41.446464343Z Gathering data for ns/openshift...
2026-09-11T02:54:41.953740255Z Gathering data for ns/kube-system...
2026-09-11T02:54:42.919926477Z Gathering data for ns/openshift-etcd...
2026-09-11T02:54:43.030998257Z Waiting on subprocesses to finish execution.
2026-09-11T02:54:43.039902354Z Executing Istio gather script
2026-09-11T02:54:43.041263929Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-11T02:54:43.051766288Z INFO: Collecting host service logs for crio
2026-09-11T02:54:43.051801888Z INFO: Waiting for node performance related collection to complete ...
2026-09-11T02:54:43.052059016Z INFO: Collecting host service logs for kubelet
2026-09-11T02:54:43.052362379Z INFO: Collecting host service logs for rpm-ostreed
2026-09-11T02:54:43.052646947Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-11T02:54:43.052969729Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-11T02:54:43.053253210Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-11T02:54:43.053573710Z INFO: Collecting host service logs for NetworkManager
2026-09-11T02:54:43.053951131Z INFO: Collecting host service logs for openvswitch
2026-09-11T02:54:43.054250803Z INFO: Collecting host service logs for ovs-configuration
2026-09-11T02:54:43.054570092Z INFO: Collecting host service logs for ovsdb-server
2026-09-11T02:54:43.054878788Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-11T02:54:43.055240726Z INFO: Waiting for worker host service log collection to complete ...
2026-09-11T02:54:43.062871125Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-11T02:54:43.065791179Z INFO: Gathering HAProxy config files
2026-09-11T02:54:43.074838827Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-11T02:54:43.318549034Z No resources found in openshift-etcd namespace.
2026-09-11T02:54:43.720195739Z No resources found
2026-09-11T02:54:43.734052561Z error: the server doesn't have a resource type "performanceprofile"
2026-09-11T02:54:43.895142132Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-11T02:54:43.977429424Z Wrote inspect data to must-gather.
2026-09-11T02:54:43.977466655Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T02:54:43.977466655Z     namespaces "assisted-installer" not found
2026-09-11T02:54:44.098895099Z INFO: Collecting Insights Archives from insights-operator-5844b856db-chq6t
2026-09-11T02:54:44.098934830Z insights-runtime-extractor-frqc7
2026-09-11T02:54:44.098955029Z insights-runtime-extractor-jdljp
2026-09-11T02:54:44.098974102Z insights-runtime-extractor-l2gb7
2026-09-11T02:54:44.284329379Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-11T02:54:44.430320540Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-11T02:54:44.438292048Z INFO: "metallb-operator" not detected. Skipping.
2026-09-11T02:54:44.454557324Z INFO: Worker host service log collection to complete.
2026-09-11T02:54:44.456317863Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-11T02:54:44.517213684Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-11T02:54:44.517301257Z INFO: on-disk MachineConfig config collection complete.
2026-09-11T02:54:44.580498009Z Wrote inspect data to must-gather.
2026-09-11T02:54:44.609801310Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-11T02:54:44.731452045Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-11T02:54:44.731452045Z See 'oc rsync -h' for help and examples
2026-09-11T02:54:44.786771453Z ERROR: No running kube-apiserver pods found
2026-09-11T02:54:44.932072918Z Wrote inspect data to must-gather.
2026-09-11T02:54:45.110069330Z Wrote inspect data to must-gather.
2026-09-11T02:54:45.195348460Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-11T02:54:45.243038072Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-11T02:54:45.287488578Z No resources found
2026-09-11T02:54:45.366233005Z Gathering data for ns/openshift-monitoring...
2026-09-11T02:54:45.376955680Z Wrote inspect data to must-gather.
2026-09-11T02:54:45.408774661Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T02:54:45.465766234Z INFO: 'previous-logs' folder not found on ip-10-0-134-12.ec2.internal, skipping...
2026-09-11T02:54:45.482024317Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-11T02:54:45.639237083Z error: the server doesn't have a resource type "machineconfigs"
2026-09-11T02:54:45.720402727Z Wrote inspect data to must-gather.
2026-09-11T02:54:45.733990060Z tar: Removing leading `/' from member names
2026-09-11T02:54:45.738758554Z INFO: HAProxy config collection complete.
2026-09-11T02:54:45.769492166Z Wrote inspect data to must-gather.
2026-09-11T02:54:45.777697385Z Wrote inspect data to must-gather.
2026-09-11T02:54:45.888013863Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-11T02:54:45.914590712Z INFO: Getting rules from prometheus-k8s-0
2026-09-11T02:54:45.929746221Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-11T02:54:45.933132113Z INFO: 'previous-logs' folder not found on ip-10-0-134-245.ec2.internal, skipping...
2026-09-11T02:54:46.057903790Z Wrote inspect data to must-gather.
2026-09-11T02:54:46.141627833Z Wrote inspect data to must-gather.
2026-09-11T02:54:46.199538990Z INFO: 'previous-logs' folder not found on ip-10-0-136-223.ec2.internal, skipping...
2026-09-11T02:54:46.199578801Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-11T02:54:46.199578801Z INFO: Machine Config Daemon termination log collection complete.
2026-09-11T02:54:46.306656626Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-11T02:54:46.322835400Z Wrote inspect data to must-gather.
2026-09-11T02:54:46.325925701Z INFO: Getting status/config from prometheus-k8s-0
2026-09-11T02:54:46.535581623Z Wrote inspect data to must-gather.
2026-09-11T02:54:46.560957175Z Wrote inspect data to must-gather.
2026-09-11T02:54:46.726493815Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-11T02:54:46.749350334Z Wrote inspect data to must-gather.
2026-09-11T02:54:46.857230538Z INFO: INTERCONNECT MODE
2026-09-11T02:54:46.857358752Z INFO: Gathering ovn-kubernetes DBs
2026-09-11T02:54:46.962547399Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-11T02:54:46.988889614Z Wrote inspect data to must-gather.
2026-09-11T02:54:46.992459695Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-11T02:54:47.041723909Z INFO: Gathering OVN_Northbound from ovnkube-node-dc82g...
2026-09-11T02:54:47.042087981Z INFO: Gathering OVN_Northbound from ovnkube-node-lztfk...
2026-09-11T02:54:47.042368141Z INFO: Gathering OVN_Northbound from ovnkube-node-xztkk...
2026-09-11T02:54:47.042686768Z INFO: Gathering OVN_Southbound from ovnkube-node-dc82g...
2026-09-11T02:54:47.043055185Z INFO: Gathering OVN_Southbound from ovnkube-node-lztfk...
2026-09-11T02:54:47.043331049Z INFO: Gathering OVN_Southbound from ovnkube-node-xztkk...
2026-09-11T02:54:47.092371697Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T02:54:47.254942644Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:47.303048986Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-11T02:54:47.329265703Z Wrote inspect data to must-gather.
2026-09-11T02:54:47.367866898Z tar: Removing leading `/' from member names
2026-09-11T02:54:47.380333911Z tar: Removing leading `/' from member names
2026-09-11T02:54:47.420409616Z tar: Removing leading `/' from member names
2026-09-11T02:54:47.441094845Z tar: Removing leading `/' from member names
2026-09-11T02:54:47.441781713Z tar: Removing leading `/' from member names
2026-09-11T02:54:47.459489781Z tar: Removing leading `/' from member names
2026-09-11T02:54:47.550942789Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:47.554337087Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-11T02:54:47.588682622Z Wrote inspect data to must-gather.
2026-09-11T02:54:47.599537233Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-11T02:54:47.796307593Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:47.850179196Z W0911 02:54:47.850106    1459 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-11T02:54:47.855708729Z Wrote inspect data to must-gather.
2026-09-11T02:54:47.895524403Z INFO: Waiting for network log collection to complete ...
2026-09-11T02:54:47.895571344Z INFO: Waiting for ovnk database copies to complete ...
2026-09-11T02:54:47.896095081Z INFO: Copying ovnk databases complete.
2026-09-11T02:54:47.901416761Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-11T02:54:47.903763164Z ovnk_database_store/
2026-09-11T02:54:47.903763164Z ovnk_database_store/ovnkube-node-lztfk_sbdb
2026-09-11T02:54:47.904619091Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T02:54:47.927247357Z ovnk_database_store/ovnkube-node-xztkk_nbdb
2026-09-11T02:54:47.935108191Z ovnk_database_store/ovnkube-node-xztkk_sbdb
2026-09-11T02:54:47.953299937Z ovnk_database_store/ovnkube-node-dc82g_nbdb
2026-09-11T02:54:47.961569375Z ovnk_database_store/ovnkube-node-dc82g_sbdb
2026-09-11T02:54:47.983866262Z ovnk_database_store/ovnkube-node-lztfk_nbdb
2026-09-11T02:54:47.995719116Z INFO: Network log collection complete.
2026-09-11T02:54:48.051030145Z INFO: Getting status from alertmanager-main-0
2026-09-11T02:54:48.051173961Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:48.053882700Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-11T02:54:48.120433373Z Gathering data for ns/openshift-network-console...
2026-09-11T02:54:48.193184668Z Wrote inspect data to must-gather.
2026-09-11T02:54:48.258846754Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:48.468852369Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-11T02:54:48.469227402Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:48.471498190Z INFO: Fallback to identify the container image from release info
2026-09-11T02:54:48.471924411Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-11T02:54:48.576160269Z Gathering data for ns/openshift-console-operator...
2026-09-11T02:54:48.701894713Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:48.877990793Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:48.880217806Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-11T02:54:48.991578368Z Gathering data for ns/openshift-console...
2026-09-11T02:54:49.048262775Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:49.069167809Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-11T02:54:49.205534059Z daemonset.apps/perf-node-gather-daemonset created
2026-09-11T02:54:49.239485902Z Wrote inspect data to /must-gather/istio.
2026-09-11T02:54:49.329672808Z Waiting for performance profile collector pods to become ready: 1
2026-09-11T02:54:49.587546284Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-11T02:54:49.940859817Z Done executing Istio gather script
2026-09-11T02:54:50.190679241Z Gathering data for ns/openshift-dns-operator...
2026-09-11T02:54:50.461871994Z Waiting for performance profile collector pods to become ready: 2
2026-09-11T02:54:50.494337881Z Gathering data for ns/openshift-dns...
2026-09-11T02:54:50.513534404Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-11T02:54:51.119363483Z Gathering data for ns/openshift-image-registry...
2026-09-11T02:54:51.578131568Z Waiting for performance profile collector pods to become ready: 3
2026-09-11T02:54:51.647727827Z Gathering data for ns/openshift-ingress-operator...
2026-09-11T02:54:51.986388206Z Gathering data for ns/openshift-ingress...
2026-09-11T02:54:52.316115734Z Gathering data for ns/openshift-ingress-canary...
2026-09-11T02:54:52.701220516Z Gathering data for ns/openshift-insights...
2026-09-11T02:54:52.702058021Z Waiting for performance profile collector pods to become ready: 4
2026-09-11T02:54:53.814037328Z Waiting for performance profile collector pods to become ready: 5
2026-09-11T02:54:54.531255877Z Gathering data for ns/openshift-jobset-operator...
2026-09-11T02:54:54.935029522Z Waiting for performance profile collector pods to become ready: 6
2026-09-11T02:54:55.327760725Z Gathering data for ns/openshift-config...
2026-09-11T02:54:55.518632537Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-11T02:54:55.647052453Z Gathering data for ns/openshift-config-managed...
2026-09-11T02:54:56.054930131Z Waiting for performance profile collector pods to become ready: 7
2026-09-11T02:54:56.109049068Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-11T02:54:56.419253646Z Gathering data for ns/openshift-kube-apiserver...
2026-09-11T02:54:56.710019659Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-11T02:54:57.015165762Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-11T02:54:57.166007775Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-11T02:54:57.377453623Z Gathering data for ns/openshift-kube-scheduler...
2026-09-11T02:54:57.418272421Z Collecting performance related data for node ip-10-0-134-245.ec2.internal
2026-09-11T02:54:57.427493341Z Collecting performance related data for node ip-10-0-136-223.ec2.internal
2026-09-11T02:54:57.444041005Z Collecting performance related data for node ip-10-0-134-12.ec2.internal
2026-09-11T02:54:57.693239338Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-11T02:54:58.078565252Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-11T02:54:58.513457577Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-11T02:54:59.048779086Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-11T02:54:59.715141872Z Gathering data for ns/openshift-multus...
2026-09-11T02:55:00.523672043Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-11T02:55:02.515137393Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-11T02:55:04.528936262Z Gathering data for ns/openshift-host-network...
2026-09-11T02:55:05.025434740Z Gathering data for ns/openshift-network-diagnostics...
2026-09-11T02:55:05.528263383Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-11T02:55:05.724173024Z Collecting kubelet logs for node ip-10-0-134-245.ec2.internal
2026-09-11T02:55:05.726549168Z Collecting kubelet logs for node ip-10-0-136-223.ec2.internal
2026-09-11T02:55:05.728758607Z Collecting kubelet logs for node ip-10-0-134-12.ec2.internal
2026-09-11T02:55:05.842050227Z Gathering data for ns/openshift-network-node-identity...
2026-09-11T02:55:06.078187469Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-p4wd8 namespace
2026-09-11T02:55:06.084759047Z INFO: Node performance data collection complete.
2026-09-11T02:55:06.358469503Z Gathering data for ns/openshift-network-operator...
2026-09-11T02:55:06.843058690Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-11T02:55:07.190686867Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-11T02:55:07.672457882Z Gathering data for ns/openshift-apiserver-operator...
2026-09-11T02:55:07.971674619Z Gathering data for ns/openshift-apiserver...
2026-09-11T02:55:08.408677626Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-11T02:55:08.735232895Z Gathering data for ns/openshift-controller-manager...
2026-09-11T02:55:09.048902349Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-11T02:55:09.742190950Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-11T02:55:10.092652879Z Gathering data for ns/openshift-service-ca-operator...
2026-09-11T02:55:10.432128320Z Gathering data for ns/openshift-service-ca...
2026-09-11T02:55:10.532920802Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-11T02:55:10.780897118Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-11T02:55:11.686655509Z Wrote inspect data to must-gather.
2026-09-11T02:55:11.686700433Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T02:55:11.686700433Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-11T02:55:11.815840756Z error: the server doesn't have a resource type "clusters"
2026-09-11T02:55:11.952688211Z Caches written to disk