2026-09-11T03:05:26.356387174Z [disk usage checker] Started
2026-09-11T03:05:26.359887769Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T03:05:26.547782251Z Gathering data for ns/openshift-cluster-version...
2026-09-11T03:05:26.687687664Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T03:05:26.771961406Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T03:05:26.849051265Z Gathering data for ns/default...
2026-09-11T03:05:27.164391657Z Gathering data for ns/openshift...
2026-09-11T03:05:27.584744367Z Gathering data for ns/kube-system...
2026-09-11T03:05:28.395993145Z Gathering data for ns/openshift-etcd...
2026-09-11T03:05:28.737410972Z Wrote inspect data to must-gather.
2026-09-11T03:05:28.737442585Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T03:05:28.737442585Z     namespaces "assisted-installer" not found
2026-09-11T03:05:28.911537839Z Waiting on subprocesses to finish execution.
2026-09-11T03:05:28.920035769Z Executing Istio gather script
2026-09-11T03:05:28.920356211Z INFO: Gathering HAProxy config files
2026-09-11T03:05:28.928476156Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-11T03:05:28.930569399Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-11T03:05:28.930966537Z INFO: Collecting host service logs for crio
2026-09-11T03:05:28.931325020Z INFO: Collecting host service logs for kubelet
2026-09-11T03:05:28.931685621Z INFO: Collecting host service logs for rpm-ostreed
2026-09-11T03:05:28.932267494Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-11T03:05:28.932618367Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-11T03:05:28.932975506Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-11T03:05:28.933327923Z INFO: Collecting host service logs for NetworkManager
2026-09-11T03:05:28.933662841Z INFO: Collecting host service logs for openvswitch
2026-09-11T03:05:28.933995831Z INFO: Collecting host service logs for ovs-configuration
2026-09-11T03:05:28.934345357Z INFO: Collecting host service logs for ovsdb-server
2026-09-11T03:05:28.934679166Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-11T03:05:28.935045192Z INFO: Waiting for worker host service log collection to complete ...
2026-09-11T03:05:28.941213855Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-11T03:05:28.969466529Z INFO: Waiting for node performance related collection to complete ...
2026-09-11T03:05:29.251272203Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-11T03:05:29.923378941Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-11T03:05:30.087579960Z INFO: "metallb-operator" not detected. Skipping.
2026-09-11T03:05:30.138900069Z error: the server doesn't have a resource type "performanceprofile"
2026-09-11T03:05:30.201771670Z No resources found
2026-09-11T03:05:30.218863169Z No resources found in openshift-etcd namespace.
2026-09-11T03:05:30.238751008Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-11T03:05:30.299793466Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-11T03:05:30.642597367Z Wrote inspect data to must-gather.
2026-09-11T03:05:30.661119178Z INFO: Worker host service log collection to complete.
2026-09-11T03:05:30.736452600Z No resources found
2026-09-11T03:05:30.789288155Z Wrote inspect data to must-gather.
2026-09-11T03:05:30.834907387Z Wrote inspect data to must-gather.
2026-09-11T03:05:30.861359949Z INFO: Collecting Insights Archives from insights-operator-5844b856db-sdd7g
2026-09-11T03:05:30.861407326Z insights-runtime-extractor-2tnk6
2026-09-11T03:05:30.861428049Z insights-runtime-extractor-7mhxg
2026-09-11T03:05:30.861449130Z insights-runtime-extractor-tj5sr
2026-09-11T03:05:30.897589228Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-11T03:05:30.898386193Z Wrote inspect data to must-gather.
2026-09-11T03:05:30.978837225Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-11T03:05:30.978917673Z INFO: on-disk MachineConfig config collection complete.
2026-09-11T03:05:31.075706220Z ERROR: No running kube-apiserver pods found
2026-09-11T03:05:31.148115860Z INFO: 'previous-logs' folder not found on ip-10-0-128-154.ec2.internal, skipping...
2026-09-11T03:05:31.164536136Z Wrote inspect data to must-gather.
2026-09-11T03:05:31.175802968Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-11T03:05:31.175802968Z See 'oc rsync -h' for help and examples
2026-09-11T03:05:31.230552201Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-11T03:05:31.258059966Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-11T03:05:31.266595638Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T03:05:31.297030065Z Wrote inspect data to must-gather.
2026-09-11T03:05:31.311904979Z Wrote inspect data to must-gather.
2026-09-11T03:05:31.394221176Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T03:05:31.436825257Z error: the server doesn't have a resource type "machineconfigs"
2026-09-11T03:05:31.554990491Z INFO: 'previous-logs' folder not found on ip-10-0-128-155.ec2.internal, skipping...
2026-09-11T03:05:31.625369561Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-11T03:05:31.655066478Z INFO: HAProxy config collection complete.
2026-09-11T03:05:31.656854970Z Gathering data for ns/openshift-monitoring...
2026-09-11T03:05:31.680366116Z Wrote inspect data to must-gather.
2026-09-11T03:05:31.819791444Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-11T03:05:31.840468558Z INFO: Getting rules from prometheus-k8s-0
2026-09-11T03:05:31.868775272Z INFO: 'previous-logs' folder not found on ip-10-0-138-232.ec2.internal, skipping...
2026-09-11T03:05:31.868900281Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-11T03:05:31.868957545Z INFO: Machine Config Daemon termination log collection complete.
2026-09-11T03:05:31.898053563Z Wrote inspect data to must-gather.
2026-09-11T03:05:31.972268462Z Wrote inspect data to must-gather.
2026-09-11T03:05:32.042440140Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-11T03:05:32.207928036Z Wrote inspect data to must-gather.
2026-09-11T03:05:32.241063746Z Wrote inspect data to must-gather.
2026-09-11T03:05:32.291840281Z INFO: Getting status/config from prometheus-k8s-0
2026-09-11T03:05:32.463995846Z Wrote inspect data to must-gather.
2026-09-11T03:05:32.479415131Z INFO: INTERCONNECT MODE
2026-09-11T03:05:32.479451191Z INFO: Gathering ovn-kubernetes DBs
2026-09-11T03:05:32.498542458Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-11T03:05:32.630722645Z INFO: Gathering OVN_Northbound from ovnkube-node-gj8bs...
2026-09-11T03:05:32.630953061Z INFO: Gathering OVN_Northbound from ovnkube-node-vcl48...
2026-09-11T03:05:32.631207868Z INFO: Gathering OVN_Northbound from ovnkube-node-vvdjq...
2026-09-11T03:05:32.631655273Z INFO: Gathering OVN_Southbound from ovnkube-node-gj8bs...
2026-09-11T03:05:32.631966090Z INFO: Gathering OVN_Southbound from ovnkube-node-vcl48...
2026-09-11T03:05:32.632288540Z INFO: Gathering OVN_Southbound from ovnkube-node-vvdjq...
2026-09-11T03:05:32.704876847Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:32.723965240Z Wrote inspect data to must-gather.
2026-09-11T03:05:32.760419988Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-11T03:05:32.985488959Z tar: Removing leading `/' from member names
2026-09-11T03:05:32.996418237Z tar: Removing leading `/' from member names
2026-09-11T03:05:33.010729055Z tar: Removing leading `/' from member names
2026-09-11T03:05:33.011823396Z tar: Removing leading `/' from member names
2026-09-11T03:05:33.012790800Z tar: Removing leading `/' from member names
2026-09-11T03:05:33.013005091Z tar: Removing leading `/' from member names
2026-09-11T03:05:33.030699085Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:33.033947459Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-11T03:05:33.080839089Z Wrote inspect data to must-gather.
2026-09-11T03:05:33.088348996Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-11T03:05:33.251461947Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:33.317779080Z Wrote inspect data to must-gather.
2026-09-11T03:05:33.401879791Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-11T03:05:33.419414162Z INFO: Waiting for network log collection to complete ...
2026-09-11T03:05:33.419451443Z INFO: Waiting for ovnk database copies to complete ...
2026-09-11T03:05:33.419934439Z INFO: Copying ovnk databases complete.
2026-09-11T03:05:33.423044293Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-09-11T03:05:33.426166356Z ovnk_database_store/
2026-09-11T03:05:33.426219388Z ovnk_database_store/ovnkube-node-vvdjq_nbdb
2026-09-11T03:05:33.432363317Z ovnk_database_store/ovnkube-node-gj8bs_sbdb
2026-09-11T03:05:33.452237103Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:33.452572403Z ovnk_database_store/ovnkube-node-vvdjq_sbdb
2026-09-11T03:05:33.454916920Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-11T03:05:33.467289965Z ovnk_database_store/ovnkube-node-vcl48_sbdb
2026-09-11T03:05:33.487378892Z ovnk_database_store/ovnkube-node-gj8bs_nbdb
2026-09-11T03:05:33.494004086Z ovnk_database_store/ovnkube-node-vcl48_nbdb
2026-09-11T03:05:33.503851960Z INFO: Network log collection complete.
2026-09-11T03:05:33.553340982Z Wrote inspect data to must-gather.
2026-09-11T03:05:33.650917991Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:33.740711418Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-11T03:05:33.750902957Z Wrote inspect data to must-gather.
2026-09-11T03:05:33.864518023Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T03:05:33.865329179Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:33.868216779Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-11T03:05:34.003859318Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-11T03:05:34.006653651Z INFO: Fallback to identify the container image from release info
2026-09-11T03:05:34.089946552Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:34.152940729Z INFO: Getting status from alertmanager-main-0
2026-09-11T03:05:34.281789924Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:34.284025485Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-11T03:05:34.431023542Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:34.486689140Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T03:05:34.619246574Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-11T03:05:34.624838263Z Wrote inspect data to /must-gather/istio.
2026-09-11T03:05:34.676221904Z Gathering data for ns/openshift-network-console...
2026-09-11T03:05:34.743814023Z daemonset.apps/perf-node-gather-daemonset created
2026-09-11T03:05:34.880570222Z Waiting for performance profile collector pods to become ready: 1
2026-09-11T03:05:35.278642112Z Gathering data for ns/openshift-console-operator...
2026-09-11T03:05:35.291293544Z Done executing Istio gather script
2026-09-11T03:05:35.714262833Z Gathering data for ns/openshift-console...
2026-09-11T03:05:35.979614734Z Waiting for performance profile collector pods to become ready: 2
2026-09-11T03:05:36.263782802Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-11T03:05:36.402783661Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T03:05:36.808371535Z Gathering data for ns/openshift-dns-operator...
2026-09-11T03:05:37.093463675Z Waiting for performance profile collector pods to become ready: 3
2026-09-11T03:05:37.183786071Z Gathering data for ns/openshift-dns...
2026-09-11T03:05:37.807106090Z Gathering data for ns/openshift-image-registry...
2026-09-11T03:05:38.199765774Z Waiting for performance profile collector pods to become ready: 4
2026-09-11T03:05:38.272038799Z Gathering data for ns/openshift-ingress-operator...
2026-09-11T03:05:38.621118967Z Gathering data for ns/openshift-ingress...
2026-09-11T03:05:39.004761635Z Gathering data for ns/openshift-ingress-canary...
2026-09-11T03:05:39.311242739Z Waiting for performance profile collector pods to become ready: 5
2026-09-11T03:05:39.428305666Z Gathering data for ns/openshift-insights...
2026-09-11T03:05:40.412695602Z Waiting for performance profile collector pods to become ready: 6
2026-09-11T03:05:41.408472578Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T03:05:41.443146432Z Gathering data for ns/openshift-config...
2026-09-11T03:05:41.535994392Z Waiting for performance profile collector pods to become ready: 7
2026-09-11T03:05:41.709777838Z Gathering data for ns/openshift-config-managed...
2026-09-11T03:05:42.093237387Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-11T03:05:42.452968701Z Gathering data for ns/openshift-kube-apiserver...
2026-09-11T03:05:42.656719948Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-11T03:05:42.720080037Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-11T03:05:42.914539766Z Collecting performance related data for node ip-10-0-138-232.ec2.internal
2026-09-11T03:05:42.922001943Z Collecting performance related data for node ip-10-0-128-155.ec2.internal
2026-09-11T03:05:42.929958283Z Collecting performance related data for node ip-10-0-128-154.ec2.internal
2026-09-11T03:05:43.046827329Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-11T03:05:43.369831290Z Gathering data for ns/openshift-kube-scheduler...
2026-09-11T03:05:43.655843264Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-11T03:05:44.013459392Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-11T03:05:44.388491672Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-11T03:05:44.711397142Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-11T03:05:45.185653021Z Gathering data for ns/openshift-multus...
2026-09-11T03:05:46.417826680Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T03:05:46.663314559Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-11T03:05:48.003198871Z Gathering data for ns/openshift-host-network...
2026-09-11T03:05:48.304382090Z Gathering data for ns/openshift-network-diagnostics...
2026-09-11T03:05:48.876504750Z Gathering data for ns/openshift-network-node-identity...
2026-09-11T03:05:49.274191566Z Gathering data for ns/openshift-network-operator...
2026-09-11T03:05:49.655978084Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-11T03:05:49.921866344Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-11T03:05:50.510473378Z Gathering data for ns/openshift-apiserver-operator...
2026-09-11T03:05:50.837123090Z Gathering data for ns/openshift-apiserver...
2026-09-11T03:05:50.960474938Z Collecting kubelet logs for node ip-10-0-138-232.ec2.internal
2026-09-11T03:05:50.962313346Z Collecting kubelet logs for node ip-10-0-128-155.ec2.internal
2026-09-11T03:05:50.964084380Z Collecting kubelet logs for node ip-10-0-128-154.ec2.internal
2026-09-11T03:05:51.297284684Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-11T03:05:51.318147721Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-xpmrb namespace
2026-09-11T03:05:51.327084305Z INFO: Node performance data collection complete.
2026-09-11T03:05:51.422837828Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-09-11T03:05:51.626396328Z Gathering data for ns/openshift-controller-manager...
2026-09-11T03:05:51.883710780Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-11T03:05:52.560063609Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-11T03:05:52.838838494Z Gathering data for ns/openshift-service-ca-operator...
2026-09-11T03:05:53.138506372Z Gathering data for ns/openshift-service-ca...
2026-09-11T03:05:53.501314782Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-11T03:05:54.375439313Z Wrote inspect data to must-gather.
2026-09-11T03:05:54.375483525Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T03:05:54.375483525Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-11T03:05:54.499877764Z error: the server doesn't have a resource type "clusters"
2026-09-11T03:05:54.619633226Z Caches written to disk