2026-07-17T18:12:33.078437378Z [disk usage checker] Started
2026-07-17T18:12:33.082023819Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-17T18:12:33.271443992Z Gathering data for ns/openshift-cluster-version...
2026-07-17T18:12:33.401361043Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-17T18:12:33.485600389Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-17T18:12:33.557791750Z Gathering data for ns/default...
2026-07-17T18:12:33.826363998Z Gathering data for ns/openshift...
2026-07-17T18:12:34.244130238Z Gathering data for ns/kube-system...
2026-07-17T18:12:34.895269772Z Gathering data for ns/openshift-etcd...
2026-07-17T18:12:35.182663123Z Wrote inspect data to must-gather.
2026-07-17T18:12:35.182694874Z error: inspection completed with the errors occurred while gathering data:
2026-07-17T18:12:35.182694874Z     namespaces "assisted-installer" not found
2026-07-17T18:12:35.503486856Z Waiting on subprocesses to finish execution.
2026-07-17T18:12:35.511048922Z Executing Istio gather script
2026-07-17T18:12:35.515487092Z INFO: Gathering HAProxy config files
2026-07-17T18:12:35.520858736Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-17T18:12:35.534546135Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-17T18:12:35.544017370Z INFO: Collecting host service logs for crio
2026-07-17T18:12:35.544337289Z INFO: Collecting host service logs for kubelet
2026-07-17T18:12:35.544678679Z INFO: Collecting host service logs for rpm-ostreed
2026-07-17T18:12:35.545020181Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-17T18:12:35.545342580Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-17T18:12:35.545678649Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-17T18:12:35.546016581Z INFO: Collecting host service logs for NetworkManager
2026-07-17T18:12:35.546333815Z INFO: Collecting host service logs for openvswitch
2026-07-17T18:12:35.546683362Z INFO: Collecting host service logs for ovs-configuration
2026-07-17T18:12:35.547020734Z INFO: Collecting host service logs for ovsdb-server
2026-07-17T18:12:35.547383049Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-17T18:12:35.547779969Z INFO: Waiting for worker host service log collection to complete ...
2026-07-17T18:12:35.553373806Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-17T18:12:35.556734114Z INFO: Waiting for node performance related collection to complete ...
2026-07-17T18:12:36.036541373Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-17T18:12:36.234155101Z No resources found
2026-07-17T18:12:36.373388800Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-17T18:12:36.405236311Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-17T18:12:36.405328826Z INFO: on-disk MachineConfig config collection complete.
2026-07-17T18:12:36.425452196Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-17T18:12:36.532389290Z INFO: "metallb-operator" not detected. Skipping.
2026-07-17T18:12:36.764590954Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-64228
2026-07-17T18:12:36.764639658Z insights-runtime-extractor-56xhg
2026-07-17T18:12:36.764660021Z insights-runtime-extractor-sc8bn
2026-07-17T18:12:36.764680222Z insights-runtime-extractor-v7s6r
2026-07-17T18:12:36.803470035Z error: the server doesn't have a resource type "performanceprofile"
2026-07-17T18:12:36.821728355Z Wrote inspect data to must-gather.
2026-07-17T18:12:36.984006368Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-17T18:12:36.998217164Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-17T18:12:37.137992531Z No resources found in openshift-etcd namespace.
2026-07-17T18:12:37.207511247Z Gathering data for ns/openshift-monitoring...
2026-07-17T18:12:37.291171071Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-17T18:12:37.428630822Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-17T18:12:37.501540709Z Wrote inspect data to must-gather.
2026-07-17T18:12:37.620135627Z No resources found
2026-07-17T18:12:37.697493567Z INFO: Worker host service log collection to complete.
2026-07-17T18:12:37.737718482Z Wrote inspect data to must-gather.
2026-07-17T18:12:37.765481189Z Wrote inspect data to must-gather.
2026-07-17T18:12:37.788930296Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-17T18:12:37.816022669Z Wrote inspect data to must-gather.
2026-07-17T18:12:37.840414631Z Wrote inspect data to must-gather.
2026-07-17T18:12:37.853526334Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-17T18:12:37.853526334Z See 'oc rsync -h' for help and examples
2026-07-17T18:12:37.874470830Z Wrote inspect data to must-gather.
2026-07-17T18:12:37.900498486Z ERROR: No running kube-apiserver pods found
2026-07-17T18:12:37.987066966Z INFO: 'previous-logs' folder not found on ip-10-0-128-79.ec2.internal, skipping...
2026-07-17T18:12:38.101754446Z error: the server doesn't have a resource type "machineconfigs"
2026-07-17T18:12:38.104957157Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-17T18:12:38.113522705Z tar: Removing leading `/' from member names
2026-07-17T18:12:38.117468306Z INFO: HAProxy config collection complete.
2026-07-17T18:12:38.160590790Z Wrote inspect data to must-gather.
2026-07-17T18:12:38.187146280Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-17T18:12:38.188502116Z INFO: Getting rules from prometheus-k8s-0
2026-07-17T18:12:38.283669574Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-17T18:12:38.303406863Z INFO: 'previous-logs' folder not found on ip-10-0-139-87.ec2.internal, skipping...
2026-07-17T18:12:38.397299233Z Wrote inspect data to must-gather.
2026-07-17T18:12:38.508525686Z Wrote inspect data to must-gather.
2026-07-17T18:12:38.563067970Z INFO: 'previous-logs' folder not found on ip-10-0-141-70.ec2.internal, skipping...
2026-07-17T18:12:38.563154185Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-17T18:12:38.563177126Z INFO: Machine Config Daemon termination log collection complete.
2026-07-17T18:12:38.591387330Z INFO: Getting status/config from prometheus-k8s-0
2026-07-17T18:12:38.665712877Z Wrote inspect data to must-gather.
2026-07-17T18:12:38.686178213Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-17T18:12:38.876627691Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-17T18:12:38.916730920Z Wrote inspect data to must-gather.
2026-07-17T18:12:38.968971134Z INFO: INTERCONNECT MODE
2026-07-17T18:12:38.969082356Z INFO: Gathering ovn-kubernetes DBs
2026-07-17T18:12:38.993424326Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-17T18:12:39.016886236Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-17T18:12:39.151464670Z INFO: Gathering OVN_Northbound from ovnkube-node-24qj7...
2026-07-17T18:12:39.151685654Z INFO: Gathering OVN_Northbound from ovnkube-node-bbs6t...
2026-07-17T18:12:39.151895951Z INFO: Gathering OVN_Northbound from ovnkube-node-bnm7v...
2026-07-17T18:12:39.152195031Z INFO: Gathering OVN_Southbound from ovnkube-node-24qj7...
2026-07-17T18:12:39.152427272Z INFO: Gathering OVN_Southbound from ovnkube-node-bbs6t...
2026-07-17T18:12:39.152770064Z INFO: Gathering OVN_Southbound from ovnkube-node-bnm7v...
2026-07-17T18:12:39.164358220Z Wrote inspect data to must-gather.
2026-07-17T18:12:39.250392864Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:39.250627540Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-17T18:12:39.447571841Z tar: Removing leading `/' from member names
2026-07-17T18:12:39.454474656Z tar: Removing leading `/' from member names
2026-07-17T18:12:39.465056310Z tar: Removing leading `/' from member names
2026-07-17T18:12:39.491810862Z tar: Removing leading `/' from member names
2026-07-17T18:12:39.508643351Z tar: Removing leading `/' from member names
2026-07-17T18:12:39.545990250Z tar: Removing leading `/' from member names
2026-07-17T18:12:39.563925801Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-17T18:12:39.566889946Z W0717 18:12:39.566846    1334 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-17T18:12:39.574109888Z Wrote inspect data to must-gather.
2026-07-17T18:12:39.601358587Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:39.604356472Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-17T18:12:39.667230244Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-17T18:12:39.812495024Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-17T18:12:39.851095542Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:39.888088342Z Gathering data for ns/openshift-network-console...
2026-07-17T18:12:39.912092186Z Wrote inspect data to must-gather.
2026-07-17T18:12:39.975116619Z INFO: Waiting for network log collection to complete ...
2026-07-17T18:12:39.975329799Z INFO: Waiting for ovnk database copies to complete ...
2026-07-17T18:12:39.975870209Z INFO: Copying ovnk databases complete.
2026-07-17T18:12:39.978172588Z 2.3M	must-gather/network_logs/ovnk_database_store
2026-07-17T18:12:39.981132272Z ovnk_database_store/
2026-07-17T18:12:39.981176590Z ovnk_database_store/ovnkube-node-24qj7_nbdb
2026-07-17T18:12:39.996258796Z ovnk_database_store/ovnkube-node-24qj7_sbdb
2026-07-17T18:12:40.014355836Z ovnk_database_store/ovnkube-node-bnm7v_nbdb
2026-07-17T18:12:40.020353763Z ovnk_database_store/ovnkube-node-bbs6t_sbdb
2026-07-17T18:12:40.036573202Z ovnk_database_store/ovnkube-node-bbs6t_nbdb
2026-07-17T18:12:40.042473634Z ovnk_database_store/ovnkube-node-bnm7v_sbdb
2026-07-17T18:12:40.059373752Z INFO: Network log collection complete.
2026-07-17T18:12:40.085427601Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:40.087445176Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-17T18:12:40.224847423Z Wrote inspect data to must-gather.
2026-07-17T18:12:40.230034393Z INFO: Getting status from alertmanager-main-0
2026-07-17T18:12:40.293665786Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:40.381582581Z Gathering data for ns/openshift-console-operator...
2026-07-17T18:12:40.506769563Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:40.508796007Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-17T18:12:40.535409111Z Wrote inspect data to must-gather.
2026-07-17T18:12:40.707863575Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:40.767250935Z Gathering data for ns/openshift-console...
2026-07-17T18:12:40.796713814Z Wrote inspect data to must-gather.
2026-07-17T18:12:40.899355062Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:40.902928727Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-17T18:12:41.041856060Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-17T18:12:41.043694787Z INFO: Fallback to identify the container image from release info
2026-07-17T18:12:41.044787976Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:41.228528256Z Wrote inspect data to /must-gather/istio.
2026-07-17T18:12:41.289597636Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-17T18:12:41.659044316Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-17T18:12:41.792148204Z Gathering data for ns/openshift-dns-operator...
2026-07-17T18:12:41.792344289Z daemonset.apps/perf-node-gather-daemonset created
2026-07-17T18:12:41.893874345Z Done executing Istio gather script
2026-07-17T18:12:41.916951210Z Waiting for performance profile collector pods to become ready: 1
2026-07-17T18:12:42.089495057Z Gathering data for ns/openshift-dns...
2026-07-17T18:12:42.703662232Z Gathering data for ns/openshift-image-registry...
2026-07-17T18:12:43.026101289Z Waiting for performance profile collector pods to become ready: 2
2026-07-17T18:12:43.116217516Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-17T18:12:43.142330875Z Gathering data for ns/openshift-ingress-operator...
2026-07-17T18:12:43.430784328Z Gathering data for ns/openshift-ingress...
2026-07-17T18:12:43.725616582Z Gathering data for ns/openshift-ingress-canary...
2026-07-17T18:12:44.081580885Z Gathering data for ns/openshift-insights...
2026-07-17T18:12:44.136038047Z Waiting for performance profile collector pods to become ready: 3
2026-07-17T18:12:45.234342301Z Waiting for performance profile collector pods to become ready: 4
2026-07-17T18:12:45.843956799Z Gathering data for ns/openshift-config...
2026-07-17T18:12:46.135793291Z Gathering data for ns/openshift-config-managed...
2026-07-17T18:12:46.334855252Z Waiting for performance profile collector pods to become ready: 5
2026-07-17T18:12:46.545188570Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-17T18:12:46.770490000Z Gathering data for ns/openshift-kube-apiserver...
2026-07-17T18:12:46.999657221Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-17T18:12:47.252395950Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-17T18:12:47.447686022Z Waiting for performance profile collector pods to become ready: 6
2026-07-17T18:12:47.513853518Z Gathering data for ns/openshift-kube-scheduler...
2026-07-17T18:12:47.744878020Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-17T18:12:48.017631374Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-17T18:12:48.121393333Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-17T18:12:48.313032030Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-17T18:12:48.562240294Z Waiting for performance profile collector pods to become ready: 7
2026-07-17T18:12:48.680106394Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-17T18:12:49.121373736Z Gathering data for ns/openshift-multus...
2026-07-17T18:12:49.663501374Z Waiting for performance profile collector pods to become ready: 8
2026-07-17T18:12:50.428730076Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-17T18:12:50.759021338Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-17T18:12:50.970021131Z Collecting performance related data for node ip-10-0-139-87.ec2.internal
2026-07-17T18:12:50.977326050Z Collecting performance related data for node ip-10-0-128-79.ec2.internal
2026-07-17T18:12:50.986986748Z Collecting performance related data for node ip-10-0-141-70.ec2.internal
2026-07-17T18:12:51.707652752Z Gathering data for ns/openshift-host-network...
2026-07-17T18:12:52.105215025Z Gathering data for ns/openshift-network-diagnostics...
2026-07-17T18:12:52.677435932Z Gathering data for ns/openshift-network-node-identity...
2026-07-17T18:12:53.052365771Z Gathering data for ns/openshift-network-operator...
2026-07-17T18:12:53.126743257Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-17T18:12:53.455352073Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-17T18:12:53.747337566Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-17T18:12:54.234714703Z Gathering data for ns/openshift-apiserver-operator...
2026-07-17T18:12:54.554808943Z Gathering data for ns/openshift-apiserver...
2026-07-17T18:12:54.911384043Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-17T18:12:55.159967023Z Gathering data for ns/openshift-controller-manager...
2026-07-17T18:12:55.438872783Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-17T18:12:56.055610254Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-17T18:12:56.338139912Z Gathering data for ns/openshift-service-ca-operator...
2026-07-17T18:12:56.634497801Z Gathering data for ns/openshift-service-ca...
2026-07-17T18:12:57.011540134Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-17T18:12:57.853802946Z Wrote inspect data to must-gather.
2026-07-17T18:12:57.853850186Z error: inspection completed with the errors occurred while gathering data:
2026-07-17T18:12:57.853850186Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-17T18:12:58.131320193Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-17T18:12:58.996041638Z Collecting kubelet logs for node ip-10-0-139-87.ec2.internal
2026-07-17T18:12:58.997813502Z Collecting kubelet logs for node ip-10-0-128-79.ec2.internal
2026-07-17T18:12:58.999517278Z Collecting kubelet logs for node ip-10-0-141-70.ec2.internal
2026-07-17T18:12:59.281338009Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-v579x namespace
2026-07-17T18:12:59.286389685Z INFO: Node performance data collection complete.
2026-07-17T18:12:59.406156732Z error: the server doesn't have a resource type "clusters"
2026-07-17T18:12:59.565545843Z Caches written to disk