2026-07-08T14:26:26.687178278Z [disk usage checker] Started
2026-07-08T14:26:26.690238272Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T14:26:26.872866091Z Gathering data for ns/openshift-cluster-version...
2026-07-08T14:26:26.995905366Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-08T14:26:27.080754652Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T14:26:27.155080441Z Gathering data for ns/default...
2026-07-08T14:26:27.419738375Z Gathering data for ns/openshift...
2026-07-08T14:26:27.792818319Z Gathering data for ns/kube-system...
2026-07-08T14:26:28.434891610Z Gathering data for ns/openshift-etcd...
2026-07-08T14:26:28.717379916Z Wrote inspect data to must-gather.
2026-07-08T14:26:28.717415236Z error: inspection completed with the errors occurred while gathering data:
2026-07-08T14:26:28.717415236Z     namespaces "assisted-installer" not found
2026-07-08T14:26:29.148847868Z Waiting on subprocesses to finish execution.
2026-07-08T14:26:29.155118444Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-08T14:26:29.161435198Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-08T14:26:29.163244196Z Executing Istio gather script
2026-07-08T14:26:29.167918993Z INFO: Gathering HAProxy config files
2026-07-08T14:26:29.182537041Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-08T14:26:29.189704569Z INFO: Waiting for node performance related collection to complete ...
2026-07-08T14:26:29.201180819Z INFO: Collecting host service logs for crio
2026-07-08T14:26:29.201834066Z INFO: Collecting host service logs for kubelet
2026-07-08T14:26:29.202194850Z INFO: Collecting host service logs for rpm-ostreed
2026-07-08T14:26:29.202526289Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-08T14:26:29.202892320Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-08T14:26:29.203235454Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-08T14:26:29.203582386Z INFO: Collecting host service logs for NetworkManager
2026-07-08T14:26:29.203909241Z INFO: Collecting host service logs for openvswitch
2026-07-08T14:26:29.204226428Z INFO: Collecting host service logs for ovs-configuration
2026-07-08T14:26:29.204533964Z INFO: Collecting host service logs for ovsdb-server
2026-07-08T14:26:29.204838466Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-08T14:26:29.205183851Z INFO: Waiting for worker host service log collection to complete ...
2026-07-08T14:26:29.858706782Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-4fth5
2026-07-08T14:26:29.858749212Z insights-runtime-extractor-5zq6s
2026-07-08T14:26:29.858771529Z insights-runtime-extractor-664kh
2026-07-08T14:26:29.858791379Z insights-runtime-extractor-wcd2g
2026-07-08T14:26:30.021196387Z INFO: "metallb-operator" not detected. Skipping.
2026-07-08T14:26:30.170000666Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-08T14:26:30.239762638Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-08T14:26:30.247466018Z INFO: on-disk MachineConfig config collection complete.
2026-07-08T14:26:30.311794644Z Wrote inspect data to must-gather.
2026-07-08T14:26:30.312240970Z No resources found in openshift-etcd namespace.
2026-07-08T14:26:30.355496983Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-08T14:26:30.363670474Z ERROR: No running kube-apiserver pods found
2026-07-08T14:26:30.388133024Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-08T14:26:30.487577337Z Wrote inspect data to must-gather.
2026-07-08T14:26:30.536631565Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-08T14:26:30.548326579Z No resources found
2026-07-08T14:26:30.690234381Z error: the server doesn't have a resource type "performanceprofile"
2026-07-08T14:26:30.785951584Z INFO: Worker host service log collection to complete.
2026-07-08T14:26:31.089802571Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-08T14:26:31.089802571Z See 'oc rsync -h' for help and examples
2026-07-08T14:26:31.182092617Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-08T14:26:31.195043832Z Wrote inspect data to must-gather.
2026-07-08T14:26:31.212581741Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-08T14:26:31.231960418Z Wrote inspect data to must-gather.
2026-07-08T14:26:31.423083031Z No resources found
2026-07-08T14:26:31.426326005Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-08T14:26:31.441151261Z Wrote inspect data to must-gather.
2026-07-08T14:26:31.463948366Z Gathering data for ns/openshift-monitoring...
2026-07-08T14:26:31.513639821Z INFO: 'previous-logs' folder not found on ip-10-0-131-246.ec2.internal, skipping...
2026-07-08T14:26:31.563085553Z Wrote inspect data to must-gather.
2026-07-08T14:26:31.618071899Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T14:26:31.667328615Z Wrote inspect data to must-gather.
2026-07-08T14:26:31.698750561Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T14:26:31.837160259Z tar: Removing leading `/' from member names
2026-07-08T14:26:31.841928157Z INFO: HAProxy config collection complete.
2026-07-08T14:26:31.850602853Z Wrote inspect data to must-gather.
2026-07-08T14:26:31.863467691Z error: the server doesn't have a resource type "machineconfigs"
2026-07-08T14:26:31.882301342Z INFO: 'previous-logs' folder not found on ip-10-0-135-131.ec2.internal, skipping...
2026-07-08T14:26:31.943228757Z INFO: Getting rules from prometheus-k8s-0
2026-07-08T14:26:31.945627574Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-08T14:26:32.046040469Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-08T14:26:32.153176815Z Wrote inspect data to must-gather.
2026-07-08T14:26:32.166776117Z INFO: 'previous-logs' folder not found on ip-10-0-136-186.ec2.internal, skipping...
2026-07-08T14:26:32.166876546Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-08T14:26:32.166928133Z INFO: Machine Config Daemon termination log collection complete.
2026-07-08T14:26:32.267610028Z Wrote inspect data to must-gather.
2026-07-08T14:26:32.369648638Z INFO: Getting status/config from prometheus-k8s-0
2026-07-08T14:26:32.396199201Z Wrote inspect data to must-gather.
2026-07-08T14:26:32.430347991Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-08T14:26:32.639667611Z Wrote inspect data to must-gather.
2026-07-08T14:26:32.654461557Z INFO: INTERCONNECT MODE
2026-07-08T14:26:32.654491739Z INFO: Gathering ovn-kubernetes DBs
2026-07-08T14:26:32.664357931Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-08T14:26:32.782658769Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-08T14:26:32.818037009Z INFO: Gathering OVN_Northbound from ovnkube-node-pd9r8...
2026-07-08T14:26:32.818243576Z INFO: Gathering OVN_Northbound from ovnkube-node-rjgf7...
2026-07-08T14:26:32.818452206Z INFO: Gathering OVN_Northbound from ovnkube-node-t4wxf...
2026-07-08T14:26:32.818772423Z INFO: Gathering OVN_Southbound from ovnkube-node-pd9r8...
2026-07-08T14:26:32.819000333Z INFO: Gathering OVN_Southbound from ovnkube-node-rjgf7...
2026-07-08T14:26:32.819310965Z INFO: Gathering OVN_Southbound from ovnkube-node-t4wxf...
2026-07-08T14:26:32.868457271Z Wrote inspect data to must-gather.
2026-07-08T14:26:32.876026619Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:33.117899949Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-08T14:26:33.203393418Z tar: Removing leading `/' from member names
2026-07-08T14:26:33.212537457Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:33.215115638Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-08T14:26:33.218811358Z Wrote inspect data to must-gather.
2026-07-08T14:26:33.231685616Z tar: Removing leading `/' from member names
2026-07-08T14:26:33.239266928Z tar: Removing leading `/' from member names
2026-07-08T14:26:33.244226322Z tar: Removing leading `/' from member names
2026-07-08T14:26:33.263018304Z tar: Removing leading `/' from member names
2026-07-08T14:26:33.293587716Z tar: Removing leading `/' from member names
2026-07-08T14:26:33.389785210Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-08T14:26:33.452533767Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:33.469855941Z Wrote inspect data to must-gather.
2026-07-08T14:26:33.615478668Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-08T14:26:33.636609110Z INFO: Waiting for network log collection to complete ...
2026-07-08T14:26:33.636655203Z INFO: Waiting for ovnk database copies to complete ...
2026-07-08T14:26:33.637135818Z INFO: Copying ovnk databases complete.
2026-07-08T14:26:33.638713256Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-08T14:26:33.641003632Z ovnk_database_store/
2026-07-08T14:26:33.641003632Z ovnk_database_store/ovnkube-node-pd9r8_nbdb
2026-07-08T14:26:33.648624021Z ovnk_database_store/ovnkube-node-rjgf7_sbdb
2026-07-08T14:26:33.666044617Z ovnk_database_store/ovnkube-node-t4wxf_nbdb
2026-07-08T14:26:33.667632158Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:33.669858592Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-08T14:26:33.671520154Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-08T14:26:33.673740528Z ovnk_database_store/ovnkube-node-rjgf7_nbdb
2026-07-08T14:26:33.680993920Z ovnk_database_store/ovnkube-node-t4wxf_sbdb
2026-07-08T14:26:33.697358459Z ovnk_database_store/ovnkube-node-pd9r8_sbdb
2026-07-08T14:26:33.698271409Z Wrote inspect data to must-gather.
2026-07-08T14:26:33.715169867Z INFO: Network log collection complete.
2026-07-08T14:26:33.874681580Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:33.909827189Z Wrote inspect data to must-gather.
2026-07-08T14:26:34.079675943Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:34.081699821Z INFO: Getting status from alertmanager-main-0
2026-07-08T14:26:34.083016431Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-08T14:26:34.125051225Z W0708 14:26:34.125007    1546 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-08T14:26:34.129944678Z Wrote inspect data to must-gather.
2026-07-08T14:26:34.233657665Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T14:26:34.317269339Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:34.384014796Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-08T14:26:34.386878722Z INFO: Fallback to identify the container image from release info
2026-07-08T14:26:34.428202263Z Gathering data for ns/openshift-network-console...
2026-07-08T14:26:34.510342667Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:34.513749160Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-08T14:26:34.658399191Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:34.835558947Z Gathering data for ns/openshift-console-operator...
2026-07-08T14:26:34.844408428Z Wrote inspect data to /must-gather/istio.
2026-07-08T14:26:35.117247116Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-08T14:26:35.256376772Z daemonset.apps/perf-node-gather-daemonset created
2026-07-08T14:26:35.283938057Z Gathering data for ns/openshift-console...
2026-07-08T14:26:35.388204439Z Waiting for performance profile collector pods to become ready: 1
2026-07-08T14:26:35.495767285Z Done executing Istio gather script
2026-07-08T14:26:35.939528971Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-08T14:26:36.472681657Z Gathering data for ns/openshift-dns-operator...
2026-07-08T14:26:36.505805196Z Waiting for performance profile collector pods to become ready: 2
2026-07-08T14:26:36.703979846Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T14:26:36.771232233Z Gathering data for ns/openshift-dns...
2026-07-08T14:26:37.406083560Z Gathering data for ns/openshift-image-registry...
2026-07-08T14:26:37.612071391Z Waiting for performance profile collector pods to become ready: 3
2026-07-08T14:26:38.013829740Z Gathering data for ns/openshift-ingress-operator...
2026-07-08T14:26:38.425444265Z Gathering data for ns/openshift-ingress...
2026-07-08T14:26:38.715354054Z Waiting for performance profile collector pods to become ready: 4
2026-07-08T14:26:38.858886375Z Gathering data for ns/openshift-ingress-canary...
2026-07-08T14:26:39.283110457Z Gathering data for ns/openshift-insights...
2026-07-08T14:26:39.851252395Z Waiting for performance profile collector pods to become ready: 5
2026-07-08T14:26:40.969837129Z Waiting for performance profile collector pods to become ready: 6
2026-07-08T14:26:41.264744122Z Gathering data for ns/openshift-config...
2026-07-08T14:26:41.516692810Z Gathering data for ns/openshift-config-managed...
2026-07-08T14:26:41.708305428Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T14:26:41.932426670Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-08T14:26:42.081671763Z Waiting for performance profile collector pods to become ready: 7
2026-07-08T14:26:42.254423518Z Gathering data for ns/openshift-kube-apiserver...
2026-07-08T14:26:42.554004659Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-08T14:26:42.890107625Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-08T14:26:43.163396479Z Gathering data for ns/openshift-kube-scheduler...
2026-07-08T14:26:43.208432082Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-08T14:26:43.438577891Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-08T14:26:43.442640415Z Collecting performance related data for node ip-10-0-135-131.ec2.internal
2026-07-08T14:26:43.450406571Z Collecting performance related data for node ip-10-0-131-246.ec2.internal
2026-07-08T14:26:43.458941403Z Collecting performance related data for node ip-10-0-136-186.ec2.internal
2026-07-08T14:26:43.798281565Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-08T14:26:44.255198195Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-08T14:26:44.623486360Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-08T14:26:45.178068607Z Gathering data for ns/openshift-multus...
2026-07-08T14:26:46.712623675Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T14:26:46.832147543Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-08T14:26:48.430265073Z Gathering data for ns/openshift-host-network...
2026-07-08T14:26:48.814052686Z Gathering data for ns/openshift-network-diagnostics...
2026-07-08T14:26:49.326702458Z Gathering data for ns/openshift-network-node-identity...
2026-07-08T14:26:49.697149372Z Gathering data for ns/openshift-network-operator...
2026-07-08T14:26:50.071318660Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-08T14:26:50.333453838Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-08T14:26:50.910327784Z Gathering data for ns/openshift-apiserver-operator...
2026-07-08T14:26:51.217067947Z Gathering data for ns/openshift-apiserver...
2026-07-08T14:26:51.347513303Z Collecting kubelet logs for node ip-10-0-135-131.ec2.internal
2026-07-08T14:26:51.350097848Z Collecting kubelet logs for node ip-10-0-131-246.ec2.internal
2026-07-08T14:26:51.351811169Z Collecting kubelet logs for node ip-10-0-136-186.ec2.internal
2026-07-08T14:26:51.643690584Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-08T14:26:51.700606622Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-2mlv9 namespace
2026-07-08T14:26:51.707913949Z INFO: Node performance data collection complete.
2026-07-08T14:26:51.717223333Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-08T14:26:51.928018097Z Gathering data for ns/openshift-controller-manager...
2026-07-08T14:26:52.195821961Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-08T14:26:52.813749080Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-08T14:26:53.121821869Z Gathering data for ns/openshift-service-ca-operator...
2026-07-08T14:26:53.418683683Z Gathering data for ns/openshift-service-ca...
2026-07-08T14:26:53.793327968Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-08T14:26:54.697753803Z Wrote inspect data to must-gather.
2026-07-08T14:26:54.697801654Z error: inspection completed with the errors occurred while gathering data:
2026-07-08T14:26:54.697801654Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-08T14:26:54.816809469Z error: the server doesn't have a resource type "clusters"
2026-07-08T14:26:54.945061592Z Caches written to disk