2026-09-07T12:23:20.873606759Z [disk usage checker] Started
2026-09-07T12:23:20.876885054Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-07T12:23:21.059887893Z Gathering data for ns/openshift-cluster-version...
2026-09-07T12:23:21.162673160Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T12:23:21.238389582Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:23:21.316298080Z Gathering data for ns/default...
2026-09-07T12:23:21.618189292Z Gathering data for ns/openshift...
2026-09-07T12:23:21.995864186Z Gathering data for ns/kube-system...
2026-09-07T12:23:22.628020832Z Gathering data for ns/openshift-etcd...
2026-09-07T12:23:22.921725580Z Wrote inspect data to must-gather.
2026-09-07T12:23:22.921813678Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T12:23:22.921813678Z     namespaces "assisted-installer" not found
2026-09-07T12:23:23.406758981Z Waiting on subprocesses to finish execution.
2026-09-07T12:23:23.413011088Z INFO: Gathering HAProxy config files
2026-09-07T12:23:23.414816783Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-07T12:23:23.416864547Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-07T12:23:23.417074239Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-07T12:23:23.425171972Z Executing Istio gather script
2026-09-07T12:23:23.431738080Z INFO: Collecting host service logs for crio
2026-09-07T12:23:23.432072994Z INFO: Collecting host service logs for kubelet
2026-09-07T12:23:23.432417510Z INFO: Collecting host service logs for rpm-ostreed
2026-09-07T12:23:23.432746761Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-07T12:23:23.433073420Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-07T12:23:23.433411951Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-07T12:23:23.433733503Z INFO: Collecting host service logs for NetworkManager
2026-09-07T12:23:23.434035110Z INFO: Collecting host service logs for openvswitch
2026-09-07T12:23:23.434350314Z INFO: Collecting host service logs for ovs-configuration
2026-09-07T12:23:23.434690560Z INFO: Collecting host service logs for ovsdb-server
2026-09-07T12:23:23.435017517Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-07T12:23:23.435419729Z INFO: Waiting for worker host service log collection to complete ...
2026-09-07T12:23:23.482190816Z INFO: Waiting for node performance related collection to complete ...
2026-09-07T12:23:24.177995009Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-07T12:23:24.253388817Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-07T12:23:24.253585392Z INFO: on-disk MachineConfig config collection complete.
2026-09-07T12:23:24.375984836Z No resources found
2026-09-07T12:23:24.396727589Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-07T12:23:24.412295130Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-msfcx
2026-09-07T12:23:24.412379234Z insights-runtime-extractor-cmb5f
2026-09-07T12:23:24.412401905Z insights-runtime-extractor-dqmsk
2026-09-07T12:23:24.412421462Z insights-runtime-extractor-jzg52
2026-09-07T12:23:24.547109088Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-07T12:23:24.551045778Z No resources found in openshift-etcd namespace.
2026-09-07T12:23:24.559925664Z Wrote inspect data to must-gather.
2026-09-07T12:23:24.600790200Z ERROR: No running kube-apiserver pods found
2026-09-07T12:23:24.741882399Z INFO: "metallb-operator" not detected. Skipping.
2026-09-07T12:23:24.833050306Z error: the server doesn't have a resource type "performanceprofile"
2026-09-07T12:23:24.890622315Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-07T12:23:24.915036884Z INFO: Worker host service log collection to complete.
2026-09-07T12:23:25.051994178Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-07T12:23:25.051994178Z See 'oc rsync -h' for help and examples
2026-09-07T12:23:25.239426180Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-07T12:23:25.249062843Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-07T12:23:25.383686117Z Wrote inspect data to must-gather.
2026-09-07T12:23:25.454451585Z No resources found
2026-09-07T12:23:25.824795351Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-07T12:23:25.869393963Z Wrote inspect data to must-gather.
2026-09-07T12:23:25.900527502Z Wrote inspect data to must-gather.
2026-09-07T12:23:25.920684563Z Gathering data for ns/openshift-monitoring...
2026-09-07T12:23:25.921734082Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:23:25.956280526Z INFO: 'previous-logs' folder not found on ip-10-0-130-85.ec2.internal, skipping...
2026-09-07T12:23:25.969962934Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-07T12:23:26.033518279Z Wrote inspect data to must-gather.
2026-09-07T12:23:26.062292416Z Wrote inspect data to must-gather.
2026-09-07T12:23:26.071915866Z Wrote inspect data to must-gather.
2026-09-07T12:23:26.076399518Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-07T12:23:26.180390240Z INFO: HAProxy config collection complete.
2026-09-07T12:23:26.187704868Z error: the server doesn't have a resource type "machineconfigs"
2026-09-07T12:23:26.308898980Z Wrote inspect data to must-gather.
2026-09-07T12:23:26.316810087Z INFO: Getting rules from prometheus-k8s-0
2026-09-07T12:23:26.323899741Z INFO: 'previous-logs' folder not found on ip-10-0-133-190.ec2.internal, skipping...
2026-09-07T12:23:26.366012539Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-07T12:23:26.579414922Z INFO: 'previous-logs' folder not found on ip-10-0-134-139.ec2.internal, skipping...
2026-09-07T12:23:26.579450494Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-07T12:23:26.579479430Z INFO: Machine Config Daemon termination log collection complete.
2026-09-07T12:23:26.580932213Z Wrote inspect data to must-gather.
2026-09-07T12:23:26.596366725Z Wrote inspect data to must-gather.
2026-09-07T12:23:26.719594651Z INFO: Getting status/config from prometheus-k8s-0
2026-09-07T12:23:26.760169036Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-07T12:23:26.840686477Z Wrote inspect data to must-gather.
2026-09-07T12:23:26.965370326Z Wrote inspect data to must-gather.
2026-09-07T12:23:27.110617591Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-07T12:23:27.121515457Z INFO: INTERCONNECT MODE
2026-09-07T12:23:27.121556127Z INFO: Gathering ovn-kubernetes DBs
2026-09-07T12:23:27.195017101Z Wrote inspect data to must-gather.
2026-09-07T12:23:27.282099024Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-07T12:23:27.311711688Z INFO: Gathering OVN_Northbound from ovnkube-node-92kzn...
2026-09-07T12:23:27.311938397Z INFO: Gathering OVN_Northbound from ovnkube-node-hc9k7...
2026-09-07T12:23:27.312181379Z INFO: Gathering OVN_Northbound from ovnkube-node-p5mrz...
2026-09-07T12:23:27.312467439Z INFO: Gathering OVN_Southbound from ovnkube-node-92kzn...
2026-09-07T12:23:27.312788147Z INFO: Gathering OVN_Southbound from ovnkube-node-hc9k7...
2026-09-07T12:23:27.313083661Z INFO: Gathering OVN_Southbound from ovnkube-node-p5mrz...
2026-09-07T12:23:27.366973232Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-07T12:23:27.383514807Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T12:23:27.516585505Z Wrote inspect data to must-gather.
2026-09-07T12:23:27.577794485Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:27.590227019Z tar: Removing leading `/' from member names
2026-09-07T12:23:27.619531060Z tar: Removing leading `/' from member names
2026-09-07T12:23:27.630496866Z tar: Removing leading `/' from member names
2026-09-07T12:23:27.666983595Z tar: Removing leading `/' from member names
2026-09-07T12:23:27.672501107Z tar: Removing leading `/' from member names
2026-09-07T12:23:27.684163414Z tar: Removing leading `/' from member names
2026-09-07T12:23:27.726277332Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-07T12:23:27.815771121Z Wrote inspect data to must-gather.
2026-09-07T12:23:27.828515231Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:27.831990833Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-07T12:23:27.978657913Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-07T12:23:28.052593052Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:28.055782248Z W0907 12:23:28.055654    1421 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-07T12:23:28.060771644Z Wrote inspect data to must-gather.
2026-09-07T12:23:28.163908426Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T12:23:28.167553923Z INFO: Waiting for network log collection to complete ...
2026-09-07T12:23:28.167581816Z INFO: Waiting for ovnk database copies to complete ...
2026-09-07T12:23:28.168045601Z INFO: Copying ovnk databases complete.
2026-09-07T12:23:28.172920525Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-07T12:23:28.177430595Z ovnk_database_store/
2026-09-07T12:23:28.177430595Z ovnk_database_store/ovnkube-node-92kzn_sbdb
2026-09-07T12:23:28.199186947Z ovnk_database_store/ovnkube-node-hc9k7_sbdb
2026-09-07T12:23:28.219291893Z ovnk_database_store/ovnkube-node-p5mrz_nbdb
2026-09-07T12:23:28.225925669Z ovnk_database_store/ovnkube-node-p5mrz_sbdb
2026-09-07T12:23:28.244286292Z ovnk_database_store/ovnkube-node-92kzn_nbdb
2026-09-07T12:23:28.252072556Z ovnk_database_store/ovnkube-node-hc9k7_nbdb
2026-09-07T12:23:28.263058983Z INFO: Network log collection complete.
2026-09-07T12:23:28.274650564Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:28.278212979Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-07T12:23:28.378285403Z Wrote inspect data to must-gather.
2026-09-07T12:23:28.387919799Z Gathering data for ns/openshift-network-console...
2026-09-07T12:23:28.407805847Z INFO: Getting status from alertmanager-main-0
2026-09-07T12:23:28.500645152Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:28.680401417Z Wrote inspect data to must-gather.
2026-09-07T12:23:28.717800880Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:28.721168341Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-07T12:23:28.825381183Z Gathering data for ns/openshift-console-operator...
2026-09-07T12:23:28.950021402Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-07T12:23:28.952088753Z INFO: Fallback to identify the container image from release info
2026-09-07T12:23:28.963390441Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:29.161727884Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:29.165146060Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-07T12:23:29.201736029Z Gathering data for ns/openshift-console...
2026-09-07T12:23:29.326387526Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:29.502785849Z Wrote inspect data to /must-gather/istio.
2026-09-07T12:23:29.654931940Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-07T12:23:29.692018317Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-07T12:23:29.796811153Z daemonset.apps/perf-node-gather-daemonset created
2026-09-07T12:23:29.918137135Z Waiting for performance profile collector pods to become ready: 1
2026-09-07T12:23:30.193662742Z Done executing Istio gather script
2026-09-07T12:23:30.211607863Z Gathering data for ns/openshift-dns-operator...
2026-09-07T12:23:30.473195900Z Gathering data for ns/openshift-dns...
2026-09-07T12:23:30.986239187Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-07T12:23:30.986889907Z Gathering data for ns/openshift-image-registry...
2026-09-07T12:23:31.034614350Z Waiting for performance profile collector pods to become ready: 2
2026-09-07T12:23:31.392410227Z Gathering data for ns/openshift-ingress-operator...
2026-09-07T12:23:31.671088958Z Gathering data for ns/openshift-ingress...
2026-09-07T12:23:31.987849662Z Gathering data for ns/openshift-ingress-canary...
2026-09-07T12:23:32.156294676Z Waiting for performance profile collector pods to become ready: 3
2026-09-07T12:23:32.361546550Z Gathering data for ns/openshift-insights...
2026-09-07T12:23:33.253691138Z Waiting for performance profile collector pods to become ready: 4
2026-09-07T12:23:34.103215455Z Gathering data for ns/openshift-jobset-operator...
2026-09-07T12:23:34.365938624Z Waiting for performance profile collector pods to become ready: 5
2026-09-07T12:23:34.897122138Z Gathering data for ns/openshift-config...
2026-09-07T12:23:35.176522707Z Gathering data for ns/openshift-config-managed...
2026-09-07T12:23:35.477554315Z Waiting for performance profile collector pods to become ready: 6
2026-09-07T12:23:35.563467310Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-07T12:23:35.790367174Z Gathering data for ns/openshift-kube-apiserver...
2026-09-07T12:23:35.991878398Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-07T12:23:36.015046718Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-07T12:23:36.276231446Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-07T12:23:36.538757003Z Gathering data for ns/openshift-kube-scheduler...
2026-09-07T12:23:36.604586829Z Waiting for performance profile collector pods to become ready: 7
2026-09-07T12:23:36.781871785Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-07T12:23:37.047892027Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-07T12:23:37.332824550Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-07T12:23:37.640060716Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-07T12:23:37.730218853Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-07T12:23:37.973630415Z Collecting performance related data for node ip-10-0-133-190.ec2.internal
2026-09-07T12:23:37.980721008Z Collecting performance related data for node ip-10-0-134-139.ec2.internal
2026-09-07T12:23:37.987296637Z Collecting performance related data for node ip-10-0-130-85.ec2.internal
2026-09-07T12:23:38.166461154Z Gathering data for ns/openshift-multus...
2026-09-07T12:23:39.638155716Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-07T12:23:40.874868152Z Gathering data for ns/openshift-host-network...
2026-09-07T12:23:40.996727073Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-07T12:23:41.190613480Z Gathering data for ns/openshift-network-diagnostics...
2026-09-07T12:23:41.671887435Z Gathering data for ns/openshift-network-node-identity...
2026-09-07T12:23:42.039603196Z Gathering data for ns/openshift-network-operator...
2026-09-07T12:23:42.387972008Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-07T12:23:42.672850268Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-07T12:23:43.138174339Z Gathering data for ns/openshift-apiserver-operator...
2026-09-07T12:23:43.425459736Z Gathering data for ns/openshift-apiserver...
2026-09-07T12:23:43.789329998Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-07T12:23:44.024124361Z Gathering data for ns/openshift-controller-manager...
2026-09-07T12:23:44.263037457Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-07T12:23:44.870782030Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-07T12:23:45.151941497Z Gathering data for ns/openshift-service-ca-operator...
2026-09-07T12:23:45.431791753Z Gathering data for ns/openshift-service-ca...
2026-09-07T12:23:45.674649899Z Collecting kubelet logs for node ip-10-0-133-190.ec2.internal
2026-09-07T12:23:45.676372533Z Collecting kubelet logs for node ip-10-0-134-139.ec2.internal
2026-09-07T12:23:45.678121689Z Collecting kubelet logs for node ip-10-0-130-85.ec2.internal
2026-09-07T12:23:45.819051097Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-07T12:23:46.001397822Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-07T12:23:46.003032085Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-zhp6q namespace
2026-09-07T12:23:46.007522697Z INFO: Node performance data collection complete.
2026-09-07T12:23:46.691043845Z Wrote inspect data to must-gather.
2026-09-07T12:23:46.691086459Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T12:23:46.691086459Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-07T12:23:46.809751650Z error: the server doesn't have a resource type "clusters"
2026-09-07T12:23:46.940685086Z Caches written to disk