2026-07-19T22:15:40.304146569Z [disk usage checker] Started
2026-07-19T22:15:40.307601100Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-19T22:15:40.498177103Z Gathering data for ns/openshift-cluster-version...
2026-07-19T22:15:40.637733476Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-19T22:15:40.721382914Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-19T22:15:40.802593181Z Gathering data for ns/default...
2026-07-19T22:15:41.072969119Z Gathering data for ns/openshift...
2026-07-19T22:15:41.466055052Z Gathering data for ns/kube-system...
2026-07-19T22:15:42.117842450Z Gathering data for ns/openshift-etcd...
2026-07-19T22:15:42.396038664Z Wrote inspect data to must-gather.
2026-07-19T22:15:42.396082955Z error: inspection completed with the errors occurred while gathering data:
2026-07-19T22:15:42.396082955Z     namespaces "assisted-installer" not found
2026-07-19T22:15:42.922739856Z Waiting on subprocesses to finish execution.
2026-07-19T22:15:42.929139478Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-19T22:15:42.931958769Z Executing Istio gather script
2026-07-19T22:15:42.941784889Z INFO: Gathering HAProxy config files
2026-07-19T22:15:42.949126779Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-19T22:15:42.951752573Z INFO: Collecting host service logs for crio
2026-07-19T22:15:42.952048247Z INFO: Collecting host service logs for kubelet
2026-07-19T22:15:42.952367866Z INFO: Collecting host service logs for rpm-ostreed
2026-07-19T22:15:42.952720461Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-19T22:15:42.953213137Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-19T22:15:42.953661827Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-19T22:15:42.953995734Z INFO: Collecting host service logs for NetworkManager
2026-07-19T22:15:42.954336902Z INFO: Collecting host service logs for openvswitch
2026-07-19T22:15:42.954700690Z INFO: Collecting host service logs for ovs-configuration
2026-07-19T22:15:42.955052302Z INFO: Collecting host service logs for ovsdb-server
2026-07-19T22:15:42.955461590Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-19T22:15:42.955954127Z INFO: Waiting for worker host service log collection to complete ...
2026-07-19T22:15:42.964922940Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-19T22:15:43.275678252Z INFO: Waiting for node performance related collection to complete ...
2026-07-19T22:15:43.330567476Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-19T22:15:43.330627761Z INFO: on-disk MachineConfig config collection complete.
2026-07-19T22:15:43.615470616Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-19T22:15:43.692249716Z Wrote inspect data to must-gather.
2026-07-19T22:15:43.785170869Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-19T22:15:43.897050442Z error: the server doesn't have a resource type "performanceprofile"
2026-07-19T22:15:43.940307838Z INFO: "metallb-operator" not detected. Skipping.
2026-07-19T22:15:44.088642664Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-9rrtl
2026-07-19T22:15:44.088697547Z insights-runtime-extractor-btgcn
2026-07-19T22:15:44.088714336Z insights-runtime-extractor-l8p9v
2026-07-19T22:15:44.088746711Z insights-runtime-extractor-ptj2t
2026-07-19T22:15:44.149250335Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-19T22:15:44.217858640Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-19T22:15:44.256625134Z ERROR: No running kube-apiserver pods found
2026-07-19T22:15:44.379367219Z Wrote inspect data to must-gather.
2026-07-19T22:15:44.434752995Z Wrote inspect data to must-gather.
2026-07-19T22:15:44.478631492Z INFO: Worker host service log collection to complete.
2026-07-19T22:15:44.490334107Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-19T22:15:44.491929858Z No resources found
2026-07-19T22:15:44.509966679Z No resources found in openshift-etcd namespace.
2026-07-19T22:15:44.696342376Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-19T22:15:44.696342376Z See 'oc rsync -h' for help and examples
2026-07-19T22:15:44.969421533Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-19T22:15:44.983857984Z Wrote inspect data to must-gather.
2026-07-19T22:15:44.985123075Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-19T22:15:45.190232195Z No resources found
2026-07-19T22:15:45.213070023Z Wrote inspect data to must-gather.
2026-07-19T22:15:45.233367337Z Wrote inspect data to must-gather.
2026-07-19T22:15:45.255551635Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-19T22:15:45.286607101Z tar: Removing leading `/' from member names
2026-07-19T22:15:45.293802592Z INFO: HAProxy config collection complete.
2026-07-19T22:15:45.318960056Z error: the server doesn't have a resource type "machineconfigs"
2026-07-19T22:15:45.327917402Z Gathering data for ns/openshift-monitoring...
2026-07-19T22:15:45.335583815Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-19T22:15:45.396070956Z INFO: 'previous-logs' folder not found on ip-10-0-129-244.ec2.internal, skipping...
2026-07-19T22:15:45.447702866Z Wrote inspect data to must-gather.
2026-07-19T22:15:45.554358457Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-19T22:15:45.716414989Z INFO: 'previous-logs' folder not found on ip-10-0-137-44.ec2.internal, skipping...
2026-07-19T22:15:45.772281683Z Wrote inspect data to must-gather.
2026-07-19T22:15:45.785091061Z INFO: Getting rules from prometheus-k8s-0
2026-07-19T22:15:45.844682741Z Wrote inspect data to must-gather.
2026-07-19T22:15:45.868562194Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-19T22:15:46.016778316Z INFO: 'previous-logs' folder not found on ip-10-0-142-196.ec2.internal, skipping...
2026-07-19T22:15:46.016881530Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-19T22:15:46.016929721Z INFO: Machine Config Daemon termination log collection complete.
2026-07-19T22:15:46.037359018Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-19T22:15:46.053681636Z Wrote inspect data to must-gather.
2026-07-19T22:15:46.180628899Z INFO: Getting status/config from prometheus-k8s-0
2026-07-19T22:15:46.260878546Z Wrote inspect data to must-gather.
2026-07-19T22:15:46.292131735Z Wrote inspect data to must-gather.
2026-07-19T22:15:46.405197971Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-19T22:15:46.471593162Z Wrote inspect data to must-gather.
2026-07-19T22:15:46.592568668Z INFO: INTERCONNECT MODE
2026-07-19T22:15:46.592700414Z INFO: Gathering ovn-kubernetes DBs
2026-07-19T22:15:46.607084332Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-19T22:15:46.608067931Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:46.716216993Z Wrote inspect data to must-gather.
2026-07-19T22:15:46.788458617Z INFO: Gathering OVN_Northbound from ovnkube-node-m77qv...
2026-07-19T22:15:46.788712767Z INFO: Gathering OVN_Northbound from ovnkube-node-qdqpn...
2026-07-19T22:15:46.788977437Z INFO: Gathering OVN_Northbound from ovnkube-node-qjgww...
2026-07-19T22:15:46.789291727Z INFO: Gathering OVN_Southbound from ovnkube-node-m77qv...
2026-07-19T22:15:46.790009319Z INFO: Gathering OVN_Southbound from ovnkube-node-qdqpn...
2026-07-19T22:15:46.790305095Z INFO: Gathering OVN_Southbound from ovnkube-node-qjgww...
2026-07-19T22:15:46.804751081Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-19T22:15:46.884761135Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-19T22:15:46.886811459Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:46.890572450Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-19T22:15:47.101229689Z tar: Removing leading `/' from member names
2026-07-19T22:15:47.130782893Z tar: Removing leading `/' from member names
2026-07-19T22:15:47.131794519Z tar: Removing leading `/' from member names
2026-07-19T22:15:47.134450803Z Wrote inspect data to must-gather.
2026-07-19T22:15:47.148120565Z tar: Removing leading `/' from member names
2026-07-19T22:15:47.160115827Z tar: Removing leading `/' from member names
2026-07-19T22:15:47.236528801Z tar: Removing leading `/' from member names
2026-07-19T22:15:47.240255426Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:47.307214369Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-19T22:15:47.383763077Z Wrote inspect data to must-gather.
2026-07-19T22:15:47.496747271Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:47.501834220Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-19T22:15:47.579169221Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-19T22:15:47.628994199Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-19T22:15:47.717770924Z INFO: Waiting for network log collection to complete ...
2026-07-19T22:15:47.717812095Z INFO: Waiting for ovnk database copies to complete ...
2026-07-19T22:15:47.718210160Z INFO: Copying ovnk databases complete.
2026-07-19T22:15:47.720190219Z 2.4M	must-gather/network_logs/ovnk_database_store
2026-07-19T22:15:47.725443485Z ovnk_database_store/
2026-07-19T22:15:47.725500882Z ovnk_database_store/ovnkube-node-qdqpn_sbdb
2026-07-19T22:15:47.728238186Z Wrote inspect data to must-gather.
2026-07-19T22:15:47.735027135Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:47.743292533Z ovnk_database_store/ovnkube-node-qdqpn_nbdb
2026-07-19T22:15:47.748376159Z ovnk_database_store/ovnkube-node-qjgww_sbdb
2026-07-19T22:15:47.763735313Z ovnk_database_store/ovnkube-node-m77qv_nbdb
2026-07-19T22:15:47.769321267Z ovnk_database_store/ovnkube-node-m77qv_sbdb
2026-07-19T22:15:47.788304370Z ovnk_database_store/ovnkube-node-qjgww_nbdb
2026-07-19T22:15:47.800627113Z INFO: Network log collection complete.
2026-07-19T22:15:47.828069203Z Gathering data for ns/openshift-network-console...
2026-07-19T22:15:47.969619137Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:47.972715685Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-19T22:15:47.998745223Z INFO: Getting status from alertmanager-main-0
2026-07-19T22:15:48.045973906Z Wrote inspect data to must-gather.
2026-07-19T22:15:48.208287048Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:48.316906389Z Gathering data for ns/openshift-console-operator...
2026-07-19T22:15:48.345465163Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-19T22:15:48.347745641Z INFO: Fallback to identify the container image from release info
2026-07-19T22:15:48.421995963Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:48.425475462Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-19T22:15:48.575321083Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:48.696263077Z Gathering data for ns/openshift-console...
2026-07-19T22:15:48.757531994Z Wrote inspect data to /must-gather/istio.
2026-07-19T22:15:49.105395344Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-19T22:15:49.184497034Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-19T22:15:49.243083268Z daemonset.apps/perf-node-gather-daemonset created
2026-07-19T22:15:49.381308859Z Waiting for performance profile collector pods to become ready: 1
2026-07-19T22:15:49.447609741Z Done executing Istio gather script
2026-07-19T22:15:49.721070834Z Gathering data for ns/openshift-dns-operator...
2026-07-19T22:15:50.018534959Z Gathering data for ns/openshift-dns...
2026-07-19T22:15:50.343740493Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-19T22:15:50.503571477Z Waiting for performance profile collector pods to become ready: 2
2026-07-19T22:15:50.583264468Z Gathering data for ns/openshift-image-registry...
2026-07-19T22:15:51.028244066Z Gathering data for ns/openshift-ingress-operator...
2026-07-19T22:15:51.313044375Z Gathering data for ns/openshift-ingress...
2026-07-19T22:15:51.628049434Z Waiting for performance profile collector pods to become ready: 3
2026-07-19T22:15:51.633112724Z Gathering data for ns/openshift-ingress-canary...
2026-07-19T22:15:51.997048384Z Gathering data for ns/openshift-insights...
2026-07-19T22:15:52.743653462Z Waiting for performance profile collector pods to become ready: 4
2026-07-19T22:15:53.789528768Z Gathering data for ns/openshift-config...
2026-07-19T22:15:53.872385750Z Waiting for performance profile collector pods to become ready: 5
2026-07-19T22:15:54.045911528Z Gathering data for ns/openshift-config-managed...
2026-07-19T22:15:54.442848861Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-19T22:15:54.725197036Z Gathering data for ns/openshift-kube-apiserver...
2026-07-19T22:15:54.966626712Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-19T22:15:55.013322704Z Waiting for performance profile collector pods to become ready: 6
2026-07-19T22:15:55.228333956Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-19T22:15:55.348569964Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-19T22:15:55.528780156Z Gathering data for ns/openshift-kube-scheduler...
2026-07-19T22:15:55.780050348Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-19T22:15:56.124355698Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-19T22:15:56.143103189Z Waiting for performance profile collector pods to become ready: 7
2026-07-19T22:15:56.449094363Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-19T22:15:56.741958743Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-19T22:15:57.212858162Z Gathering data for ns/openshift-multus...
2026-07-19T22:15:57.259031450Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-19T22:15:57.477215331Z Collecting performance related data for node ip-10-0-137-44.ec2.internal
2026-07-19T22:15:57.485025898Z Collecting performance related data for node ip-10-0-129-244.ec2.internal
2026-07-19T22:15:57.492519364Z Collecting performance related data for node ip-10-0-142-196.ec2.internal
2026-07-19T22:15:58.699762724Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-19T22:15:59.938805229Z Gathering data for ns/openshift-host-network...
2026-07-19T22:16:00.260818928Z Gathering data for ns/openshift-network-diagnostics...
2026-07-19T22:16:00.353276989Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-19T22:16:00.798343526Z Gathering data for ns/openshift-network-node-identity...
2026-07-19T22:16:01.203821636Z Gathering data for ns/openshift-network-operator...
2026-07-19T22:16:01.742906419Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-19T22:16:02.054242844Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-19T22:16:02.569984587Z Gathering data for ns/openshift-apiserver-operator...
2026-07-19T22:16:02.875789095Z Gathering data for ns/openshift-apiserver...
2026-07-19T22:16:03.303875122Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-19T22:16:03.597010905Z Gathering data for ns/openshift-controller-manager...
2026-07-19T22:16:03.876557644Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-19T22:16:04.560921544Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-19T22:16:04.858104204Z Gathering data for ns/openshift-service-ca-operator...
2026-07-19T22:16:05.196192696Z Gathering data for ns/openshift-service-ca...
2026-07-19T22:16:05.358168533Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-19T22:16:05.568356654Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-19T22:16:05.629961003Z Collecting kubelet logs for node ip-10-0-137-44.ec2.internal
2026-07-19T22:16:05.631904015Z Collecting kubelet logs for node ip-10-0-129-244.ec2.internal
2026-07-19T22:16:05.633811155Z Collecting kubelet logs for node ip-10-0-142-196.ec2.internal
2026-07-19T22:16:05.963651998Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-x6n6h namespace
2026-07-19T22:16:05.968482320Z INFO: Node performance data collection complete.
2026-07-19T22:16:06.509161219Z Wrote inspect data to must-gather.
2026-07-19T22:16:06.509215536Z error: inspection completed with the errors occurred while gathering data:
2026-07-19T22:16:06.509215536Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-19T22:16:06.630007260Z error: the server doesn't have a resource type "clusters"
2026-07-19T22:16:06.762967431Z Caches written to disk