2026-07-29T12:00:14.236508859Z [disk usage checker] Started
2026-07-29T12:00:14.240083262Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-07-29T12:00:14.622810553Z Gathering data for ns/openshift-cluster-version...
2026-07-29T12:00:14.778101224Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T12:00:14.862802033Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T12:00:14.938331533Z Gathering data for ns/default...
2026-07-29T12:00:15.222241431Z Gathering data for ns/openshift...
2026-07-29T12:00:15.611114153Z Gathering data for ns/kube-system...
2026-07-29T12:00:16.306752128Z Gathering data for ns/openshift-etcd...
2026-07-29T12:00:16.592872815Z Wrote inspect data to must-gather.
2026-07-29T12:00:16.592905607Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T12:00:16.592905607Z     namespaces "assisted-installer" not found
2026-07-29T12:00:17.131050295Z Waiting on subprocesses to finish execution.
2026-07-29T12:00:17.142550028Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-29T12:00:17.142775625Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-29T12:00:17.146611727Z INFO: Gathering HAProxy config files
2026-07-29T12:00:17.149408114Z INFO: Collecting host service logs for crio
2026-07-29T12:00:17.149730228Z INFO: Collecting host service logs for kubelet
2026-07-29T12:00:17.150062929Z INFO: Collecting host service logs for rpm-ostreed
2026-07-29T12:00:17.150404949Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-29T12:00:17.150725321Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-29T12:00:17.151035285Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-29T12:00:17.151395562Z INFO: Collecting host service logs for NetworkManager
2026-07-29T12:00:17.151727173Z INFO: Collecting host service logs for openvswitch
2026-07-29T12:00:17.152058147Z INFO: Collecting host service logs for ovs-configuration
2026-07-29T12:00:17.152404159Z INFO: Collecting host service logs for ovsdb-server
2026-07-29T12:00:17.152752515Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-29T12:00:17.153280927Z INFO: Waiting for worker host service log collection to complete ...
2026-07-29T12:00:17.161654441Z Executing Istio gather script
2026-07-29T12:00:17.166358600Z INFO: Waiting for node performance related collection to complete ...
2026-07-29T12:00:17.192480754Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-29T12:00:17.587452810Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-29T12:00:17.876161664Z No resources found in openshift-etcd namespace.
2026-07-29T12:00:17.946543868Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-29T12:00:17.946628124Z INFO: on-disk MachineConfig config collection complete.
2026-07-29T12:00:18.232490829Z No resources found
2026-07-29T12:00:18.308500452Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-29T12:00:18.380118472Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-29T12:00:18.417171603Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-29T12:00:18.427490260Z INFO: "metallb-operator" not detected. Skipping.
2026-07-29T12:00:18.703040184Z error: the server doesn't have a resource type "performanceprofile"
2026-07-29T12:00:18.714020481Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-ktbzq
2026-07-29T12:00:18.714086531Z insights-runtime-extractor-ccdw4
2026-07-29T12:00:18.714107327Z insights-runtime-extractor-fmvfh
2026-07-29T12:00:18.714144729Z insights-runtime-extractor-r785v
2026-07-29T12:00:18.767634399Z INFO: Worker host service log collection to complete.
2026-07-29T12:00:18.808552987Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-29T12:00:18.809094728Z Wrote inspect data to must-gather.
2026-07-29T12:00:18.832645135Z Wrote inspect data to must-gather.
2026-07-29T12:00:19.018976094Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-29T12:00:19.018976094Z See 'oc rsync -h' for help and examples
2026-07-29T12:00:19.109730438Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-29T12:00:19.300277813Z ERROR: No running kube-apiserver pods found
2026-07-29T12:00:19.326916059Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-07-29T12:00:19.396203239Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-29T12:00:19.724999483Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T12:00:19.761255077Z Wrote inspect data to must-gather.
2026-07-29T12:00:19.786199675Z No resources found
2026-07-29T12:00:19.826525716Z Gathering data for ns/openshift-monitoring...
2026-07-29T12:00:19.846666228Z Wrote inspect data to must-gather.
2026-07-29T12:00:19.868908947Z tar: Removing leading `/' from member names
2026-07-29T12:00:19.878564920Z INFO: HAProxy config collection complete.
2026-07-29T12:00:19.907555533Z Wrote inspect data to must-gather.
2026-07-29T12:00:20.032023101Z INFO: 'previous-logs' folder not found on ip-10-0-134-91.ec2.internal, skipping...
2026-07-29T12:00:20.061650635Z Wrote inspect data to must-gather.
2026-07-29T12:00:20.116898552Z INFO: Getting rules from prometheus-k8s-0
2026-07-29T12:00:20.135992546Z error: the server doesn't have a resource type "machineconfigs"
2026-07-29T12:00:20.140865644Z Wrote inspect data to must-gather.
2026-07-29T12:00:20.373972356Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-29T12:00:20.398442009Z INFO: 'previous-logs' folder not found on ip-10-0-138-159.ec2.internal, skipping...
2026-07-29T12:00:20.410851985Z Wrote inspect data to must-gather.
2026-07-29T12:00:20.680572999Z Wrote inspect data to must-gather.
2026-07-29T12:00:20.722998709Z INFO: Getting status/config from prometheus-k8s-0
2026-07-29T12:00:20.752678571Z INFO: 'previous-logs' folder not found on ip-10-0-138-52.ec2.internal, skipping...
2026-07-29T12:00:20.752726014Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-29T12:00:20.752742052Z INFO: Machine Config Daemon termination log collection complete.
2026-07-29T12:00:20.754105433Z Wrote inspect data to must-gather.
2026-07-29T12:00:20.888648413Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-29T12:00:20.930846448Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-29T12:00:21.017752825Z Wrote inspect data to must-gather.
2026-07-29T12:00:21.116215949Z Wrote inspect data to must-gather.
2026-07-29T12:00:21.179219895Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-29T12:00:21.353001329Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T12:00:21.357377651Z INFO: INTERCONNECT MODE
2026-07-29T12:00:21.357406167Z INFO: Gathering ovn-kubernetes DBs
2026-07-29T12:00:21.384818155Z Wrote inspect data to must-gather.
2026-07-29T12:00:21.421545343Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-29T12:00:21.443344903Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-29T12:00:21.559796290Z INFO: Gathering OVN_Northbound from ovnkube-node-k49lk...
2026-07-29T12:00:21.560100689Z INFO: Gathering OVN_Northbound from ovnkube-node-ms96z...
2026-07-29T12:00:21.560529975Z INFO: Gathering OVN_Northbound from ovnkube-node-txz46...
2026-07-29T12:00:21.560858916Z INFO: Gathering OVN_Southbound from ovnkube-node-k49lk...
2026-07-29T12:00:21.561189779Z INFO: Gathering OVN_Southbound from ovnkube-node-ms96z...
2026-07-29T12:00:21.561500046Z INFO: Gathering OVN_Southbound from ovnkube-node-txz46...
2026-07-29T12:00:21.695331399Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:21.699861666Z Wrote inspect data to must-gather.
2026-07-29T12:00:21.805213948Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-29T12:00:21.873718090Z tar: Removing leading `/' from member names
2026-07-29T12:00:21.919776442Z tar: Removing leading `/' from member names
2026-07-29T12:00:21.953292443Z tar: Removing leading `/' from member names
2026-07-29T12:00:21.953292443Z tar: Removing leading `/' from member names
2026-07-29T12:00:21.959518472Z tar: Removing leading `/' from member names
2026-07-29T12:00:21.993014854Z tar: Removing leading `/' from member names
2026-07-29T12:00:22.068364738Z W0729 12:00:22.068316    1376 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-29T12:00:22.074536693Z Wrote inspect data to must-gather.
2026-07-29T12:00:22.091399402Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:22.094651027Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-29T12:00:22.129479598Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-29T12:00:22.185181624Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T12:00:22.369781729Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:22.465701408Z Gathering data for ns/openshift-network-console...
2026-07-29T12:00:22.472232478Z Wrote inspect data to must-gather.
2026-07-29T12:00:22.555569309Z INFO: Waiting for network log collection to complete ...
2026-07-29T12:00:22.555614295Z INFO: Waiting for ovnk database copies to complete ...
2026-07-29T12:00:22.556109404Z INFO: Copying ovnk databases complete.
2026-07-29T12:00:22.562951337Z 4.6M	must-gather/network_logs/ovnk_database_store
2026-07-29T12:00:22.566058660Z ovnk_database_store/
2026-07-29T12:00:22.566109521Z ovnk_database_store/ovnkube-node-txz46_nbdb
2026-07-29T12:00:22.581690503Z ovnk_database_store/ovnkube-node-ms96z_nbdb
2026-07-29T12:00:22.593260943Z ovnk_database_store/ovnkube-node-ms96z_sbdb
2026-07-29T12:00:22.621497659Z ovnk_database_store/ovnkube-node-txz46_sbdb
2026-07-29T12:00:22.631645663Z INFO: Getting status from alertmanager-main-0
2026-07-29T12:00:22.636752477Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:22.640362601Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-29T12:00:22.649453885Z ovnk_database_store/ovnkube-node-k49lk_nbdb
2026-07-29T12:00:22.663763303Z ovnk_database_store/ovnkube-node-k49lk_sbdb
2026-07-29T12:00:22.696650079Z INFO: Network log collection complete.
2026-07-29T12:00:22.817519871Z Wrote inspect data to must-gather.
2026-07-29T12:00:22.873347286Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:22.997034171Z Gathering data for ns/openshift-console-operator...
2026-07-29T12:00:23.106284324Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:23.110174854Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-29T12:00:23.133685416Z Wrote inspect data to must-gather.
2026-07-29T12:00:23.342647087Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:23.403175692Z Gathering data for ns/openshift-console...
2026-07-29T12:00:23.405025620Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-29T12:00:23.407995845Z INFO: Fallback to identify the container image from release info
2026-07-29T12:00:23.573966237Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:23.577552759Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-29T12:00:23.736263893Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:23.902841660Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-29T12:00:23.928276707Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:23.932103659Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-29T12:00:24.119861728Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-29T12:00:24.125572312Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:24.263993191Z daemonset.apps/perf-node-gather-daemonset created
2026-07-29T12:00:24.354884450Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:24.359046360Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-07-29T12:00:24.359102642Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-29T12:00:24.418289161Z Waiting for performance profile collector pods to become ready: 1
2026-07-29T12:00:24.453099185Z Gathering data for ns/openshift-dns-operator...
2026-07-29T12:00:24.547596685Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:24.778390379Z Wrote inspect data to /must-gather/istio.
2026-07-29T12:00:24.793591100Z Gathering data for ns/openshift-dns...
2026-07-29T12:00:25.356695707Z Gathering data for ns/openshift-image-registry...
2026-07-29T12:00:25.446924443Z Done executing Istio gather script
2026-07-29T12:00:25.549381012Z Waiting for performance profile collector pods to become ready: 2
2026-07-29T12:00:25.825024313Z Gathering data for ns/openshift-ingress-operator...
2026-07-29T12:00:26.114017017Z Gathering data for ns/openshift-ingress...
2026-07-29T12:00:26.423840924Z Gathering data for ns/openshift-ingress-canary...
2026-07-29T12:00:26.664765449Z Waiting for performance profile collector pods to become ready: 3
2026-07-29T12:00:26.774721839Z Gathering data for ns/openshift-insights...
2026-07-29T12:00:27.788076328Z Waiting for performance profile collector pods to become ready: 4
2026-07-29T12:00:28.898545708Z Waiting for performance profile collector pods to become ready: 5
2026-07-29T12:00:29.019356250Z Gathering data for ns/kserve...
2026-07-29T12:00:29.364880073Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-07-29T12:00:30.010543110Z Waiting for performance profile collector pods to become ready: 6
2026-07-29T12:00:31.136298492Z Waiting for performance profile collector pods to become ready: 7
2026-07-29T12:00:31.382465510Z Gathering data for ns/openshift-config...
2026-07-29T12:00:31.669457172Z Gathering data for ns/openshift-config-managed...
2026-07-29T12:00:32.060569515Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-29T12:00:32.248109575Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-29T12:00:32.361584484Z Gathering data for ns/openshift-kube-apiserver...
2026-07-29T12:00:32.523183593Z Collecting performance related data for node ip-10-0-138-159.ec2.internal
2026-07-29T12:00:32.529829611Z Collecting performance related data for node ip-10-0-134-91.ec2.internal
2026-07-29T12:00:32.538450386Z Collecting performance related data for node ip-10-0-138-52.ec2.internal
2026-07-29T12:00:32.668788286Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-29T12:00:32.992341516Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-29T12:00:33.339510553Z Gathering data for ns/openshift-kube-scheduler...
2026-07-29T12:00:33.677359155Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-29T12:00:34.042952472Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-29T12:00:34.352150756Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-29T12:00:34.369598586Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-07-29T12:00:34.634117788Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-29T12:00:35.149998027Z Gathering data for ns/openshift-multus...
2026-07-29T12:00:36.493209769Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-29T12:00:37.636228674Z Gathering data for ns/openshift-host-network...
2026-07-29T12:00:37.934763646Z Gathering data for ns/openshift-network-diagnostics...
2026-07-29T12:00:38.411431630Z Gathering data for ns/openshift-network-node-identity...
2026-07-29T12:00:38.805859691Z Gathering data for ns/openshift-network-operator...
2026-07-29T12:00:39.155083806Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-29T12:00:39.374112932Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-07-29T12:00:39.394534438Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-29T12:00:39.862529232Z Gathering data for ns/openshift-apiserver-operator...
2026-07-29T12:00:40.136299723Z Gathering data for ns/openshift-apiserver...
2026-07-29T12:00:40.608797831Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-29T12:00:40.791914663Z Collecting kubelet logs for node ip-10-0-138-159.ec2.internal
2026-07-29T12:00:40.794192660Z Collecting kubelet logs for node ip-10-0-134-91.ec2.internal
2026-07-29T12:00:40.796380952Z Collecting kubelet logs for node ip-10-0-138-52.ec2.internal
2026-07-29T12:00:40.873494771Z Gathering data for ns/openshift-controller-manager...
2026-07-29T12:00:41.191240233Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-29T12:00:41.303431527Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-fjdcv namespace
2026-07-29T12:00:41.310894829Z INFO: Node performance data collection complete.
2026-07-29T12:00:41.824157380Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-29T12:00:42.092169042Z Gathering data for ns/openshift-service-ca-operator...
2026-07-29T12:00:42.380683931Z Gathering data for ns/openshift-service-ca...
2026-07-29T12:00:42.705923975Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-29T12:00:43.503980578Z Wrote inspect data to must-gather.
2026-07-29T12:00:43.504023624Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T12:00:43.504023624Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-29T12:00:43.630525904Z error: the server doesn't have a resource type "clusters"
2026-07-29T12:00:43.794393258Z Caches written to disk