2026-06-09T16:14:16.232296733Z [disk usage checker] Started
2026-06-09T16:14:16.236163136Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-09T16:14:16.429873329Z Gathering data for ns/openshift-cluster-version...
2026-06-09T16:14:16.552368529Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T16:14:16.639917987Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T16:14:16.719122008Z Gathering data for ns/default...
2026-06-09T16:14:16.986564153Z Gathering data for ns/openshift...
2026-06-09T16:14:17.409200496Z Gathering data for ns/kube-system...
2026-06-09T16:14:18.080552097Z Gathering data for ns/openshift-etcd...
2026-06-09T16:14:18.390941773Z Wrote inspect data to must-gather.
2026-06-09T16:14:18.391066375Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T16:14:18.391066375Z     namespaces "assisted-installer" not found
2026-06-09T16:14:18.854214000Z Waiting on subprocesses to finish execution.
2026-06-09T16:14:18.865091411Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-09T16:14:18.867510923Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-09T16:14:18.869851545Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-09T16:14:18.870460159Z Executing Istio gather script
2026-06-09T16:14:18.877205572Z INFO: Gathering HAProxy config files
2026-06-09T16:14:18.887130223Z INFO: Collecting host service logs for crio
2026-06-09T16:14:18.887513085Z INFO: Collecting host service logs for kubelet
2026-06-09T16:14:18.887831556Z INFO: Collecting host service logs for rpm-ostreed
2026-06-09T16:14:18.888146052Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-09T16:14:18.888438333Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-09T16:14:18.888732669Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-09T16:14:18.889129944Z INFO: Collecting host service logs for NetworkManager
2026-06-09T16:14:18.889437736Z INFO: Collecting host service logs for openvswitch
2026-06-09T16:14:18.889738559Z INFO: Collecting host service logs for ovs-configuration
2026-06-09T16:14:18.890050269Z INFO: Collecting host service logs for ovsdb-server
2026-06-09T16:14:18.890355256Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-09T16:14:18.890717320Z INFO: Waiting for worker host service log collection to complete ...
2026-06-09T16:14:19.114986560Z INFO: Waiting for node performance related collection to complete ...
2026-06-09T16:14:19.636888506Z No resources found
2026-06-09T16:14:19.708957374Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-09T16:14:19.839948059Z INFO: Collecting Insights Archives from insights-operator-896549976-bf5np
2026-06-09T16:14:19.840060438Z insights-runtime-extractor-2dmfq
2026-06-09T16:14:19.840085868Z insights-runtime-extractor-7cvd7
2026-06-09T16:14:19.840106632Z insights-runtime-extractor-mkbjj
2026-06-09T16:14:19.887745816Z No resources found in openshift-etcd namespace.
2026-06-09T16:14:20.020525326Z INFO: "metallb-operator" not detected. Skipping.
2026-06-09T16:14:20.227258324Z Wrote inspect data to must-gather.
2026-06-09T16:14:20.294309960Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-09T16:14:20.294309960Z See 'oc rsync -h' for help and examples
2026-06-09T16:14:20.366792626Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-09T16:14:20.387932823Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-09T16:14:20.387982703Z INFO: on-disk MachineConfig config collection complete.
2026-06-09T16:14:20.443067970Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-09T16:14:20.643402446Z Wrote inspect data to must-gather.
2026-06-09T16:14:20.672058121Z Wrote inspect data to must-gather.
2026-06-09T16:14:20.732852387Z error: the server doesn't have a resource type "performanceprofile"
2026-06-09T16:14:20.790049761Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-09T16:14:20.822963796Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-09T16:14:21.023841011Z INFO: Worker host service log collection to complete.
2026-06-09T16:14:21.062009840Z Wrote inspect data to must-gather.
2026-06-09T16:14:21.089899533Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-09T16:14:21.150403517Z No resources found
2026-06-09T16:14:21.165211652Z ERROR: No running kube-apiserver pods found
2026-06-09T16:14:21.194663371Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-09T16:14:21.250260956Z Wrote inspect data to must-gather.
2026-06-09T16:14:21.251160985Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-09T16:14:21.251911814Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-09T16:14:21.255954335Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T16:14:21.338160995Z tar: Removing leading `/' from member names
2026-06-09T16:14:21.343650657Z INFO: HAProxy config collection complete.
2026-06-09T16:14:21.378676568Z Wrote inspect data to must-gather.
2026-06-09T16:14:21.408934033Z INFO: 'previous-logs' folder not found on ip-10-0-128-73.ec2.internal, skipping...
2026-06-09T16:14:21.480678784Z Wrote inspect data to must-gather.
2026-06-09T16:14:21.554813019Z Gathering data for ns/openshift-monitoring...
2026-06-09T16:14:21.580288762Z Wrote inspect data to must-gather.
2026-06-09T16:14:21.683016536Z error: the server doesn't have a resource type "machineconfigs"
2026-06-09T16:14:21.706922624Z INFO: 'previous-logs' folder not found on ip-10-0-129-94.ec2.internal, skipping...
2026-06-09T16:14:21.725321366Z INFO: Getting rules from prometheus-k8s-0
2026-06-09T16:14:21.865407480Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-09T16:14:21.869312457Z Wrote inspect data to must-gather.
2026-06-09T16:14:21.990362722Z INFO: 'previous-logs' folder not found on ip-10-0-136-159.ec2.internal, skipping...
2026-06-09T16:14:21.990407045Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-09T16:14:21.990407045Z INFO: Machine Config Daemon termination log collection complete.
2026-06-09T16:14:22.105701715Z Wrote inspect data to must-gather.
2026-06-09T16:14:22.131918197Z Wrote inspect data to must-gather.
2026-06-09T16:14:22.149372867Z INFO: Getting status/config from prometheus-k8s-0
2026-06-09T16:14:22.263041349Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-09T16:14:22.398337285Z INFO: INTERCONNECT MODE
2026-06-09T16:14:22.398366056Z INFO: Gathering ovn-kubernetes DBs
2026-06-09T16:14:22.474225316Z Wrote inspect data to must-gather.
2026-06-09T16:14:22.536355691Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-09T16:14:22.561386665Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-09T16:14:22.561706975Z INFO: Gathering OVN_Northbound from ovnkube-node-fn5tm...
2026-06-09T16:14:22.561957993Z INFO: Gathering OVN_Northbound from ovnkube-node-hwnns...
2026-06-09T16:14:22.562228631Z INFO: Gathering OVN_Northbound from ovnkube-node-pqzgm...
2026-06-09T16:14:22.562513869Z INFO: Gathering OVN_Southbound from ovnkube-node-fn5tm...
2026-06-09T16:14:22.562795482Z INFO: Gathering OVN_Southbound from ovnkube-node-hwnns...
2026-06-09T16:14:22.563136157Z INFO: Gathering OVN_Southbound from ovnkube-node-pqzgm...
2026-06-09T16:14:22.787955395Z Wrote inspect data to must-gather.
2026-06-09T16:14:22.882559975Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:22.892236187Z tar: Removing leading `/' from member names
2026-06-09T16:14:22.908882642Z tar: Removing leading `/' from member names
2026-06-09T16:14:22.923238856Z tar: Removing leading `/' from member names
2026-06-09T16:14:22.927502948Z tar: Removing leading `/' from member names
2026-06-09T16:14:22.962628530Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-09T16:14:22.995459133Z tar: Removing leading `/' from member names
2026-06-09T16:14:23.001972957Z tar: Removing leading `/' from member names
2026-06-09T16:14:23.070896115Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T16:14:23.141251104Z Wrote inspect data to must-gather.
2026-06-09T16:14:23.164401992Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:23.167457302Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-09T16:14:23.225270334Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-09T16:14:23.396371992Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:23.402704620Z Wrote inspect data to must-gather.
2026-06-09T16:14:23.486269939Z INFO: Waiting for network log collection to complete ...
2026-06-09T16:14:23.486388584Z INFO: Waiting for ovnk database copies to complete ...
2026-06-09T16:14:23.486909160Z INFO: Copying ovnk databases complete.
2026-06-09T16:14:23.488674842Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-06-09T16:14:23.491089173Z ovnk_database_store/
2026-06-09T16:14:23.491089173Z ovnk_database_store/ovnkube-node-fn5tm_sbdb
2026-06-09T16:14:23.491629487Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-09T16:14:23.522059976Z ovnk_database_store/ovnkube-node-hwnns_nbdb
2026-06-09T16:14:23.535807381Z ovnk_database_store/ovnkube-node-pqzgm_sbdb
2026-06-09T16:14:23.562406492Z ovnk_database_store/ovnkube-node-fn5tm_nbdb
2026-06-09T16:14:23.572040660Z ovnk_database_store/ovnkube-node-pqzgm_nbdb
2026-06-09T16:14:23.582562229Z ovnk_database_store/ovnkube-node-hwnns_sbdb
2026-06-09T16:14:23.607096319Z INFO: Network log collection complete.
2026-06-09T16:14:23.627342733Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:23.632057934Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-09T16:14:23.650226765Z W0609 16:14:23.650184    1486 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-09T16:14:23.655515067Z Wrote inspect data to must-gather.
2026-06-09T16:14:23.736741739Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T16:14:23.819307784Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:23.900142514Z INFO: Getting status from alertmanager-main-0
2026-06-09T16:14:23.938082848Z Gathering data for ns/openshift-network-console...
2026-06-09T16:14:23.960196852Z Wrote inspect data to must-gather.
2026-06-09T16:14:24.044246173Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:24.047957171Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-09T16:14:24.269740781Z Wrote inspect data to must-gather.
2026-06-09T16:14:24.276702332Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:24.401698606Z Gathering data for ns/openshift-console-operator...
2026-06-09T16:14:24.499124321Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:24.502489159Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-09T16:14:24.547068683Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-09T16:14:24.549249665Z INFO: Fallback to identify the container image from release info
2026-06-09T16:14:24.673744706Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:24.804009641Z Gathering data for ns/openshift-console...
2026-06-09T16:14:24.865480056Z Wrote inspect data to /must-gather/istio.
2026-06-09T16:14:25.302145736Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-09T16:14:25.372377817Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-09T16:14:25.517405197Z daemonset.apps/perf-node-gather-daemonset created
2026-06-09T16:14:25.566179391Z Done executing Istio gather script
2026-06-09T16:14:25.648206300Z Waiting for performance profile collector pods to become ready: 1
2026-06-09T16:14:25.820582754Z Gathering data for ns/openshift-dns-operator...
2026-06-09T16:14:26.102358704Z Gathering data for ns/openshift-dns...
2026-06-09T16:14:26.259230789Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-09T16:14:26.651786445Z Gathering data for ns/openshift-image-registry...
2026-06-09T16:14:26.755440705Z Waiting for performance profile collector pods to become ready: 2
2026-06-09T16:14:27.099191155Z Gathering data for ns/openshift-ingress-operator...
2026-06-09T16:14:27.407833959Z Gathering data for ns/openshift-ingress...
2026-06-09T16:14:27.711125407Z Gathering data for ns/openshift-ingress-canary...
2026-06-09T16:14:27.883182954Z Waiting for performance profile collector pods to become ready: 3
2026-06-09T16:14:28.085371697Z Gathering data for ns/openshift-insights...
2026-06-09T16:14:28.986348789Z Waiting for performance profile collector pods to become ready: 4
2026-06-09T16:14:30.052726239Z Gathering data for ns/openshift-config...
2026-06-09T16:14:30.110284549Z Waiting for performance profile collector pods to become ready: 5
2026-06-09T16:14:30.293636465Z Gathering data for ns/openshift-config-managed...
2026-06-09T16:14:30.665712789Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-09T16:14:30.943640885Z Gathering data for ns/openshift-kube-apiserver...
2026-06-09T16:14:31.187175781Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-09T16:14:31.246084598Z Waiting for performance profile collector pods to become ready: 6
2026-06-09T16:14:31.263814948Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-09T16:14:31.458813623Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-09T16:14:31.715233700Z Gathering data for ns/openshift-kube-scheduler...
2026-06-09T16:14:31.950723949Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-09T16:14:32.330870818Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-09T16:14:32.364870434Z Waiting for performance profile collector pods to become ready: 7
2026-06-09T16:14:32.638031564Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-09T16:14:33.009795174Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-09T16:14:33.470603645Z Gathering data for ns/openshift-multus...
2026-06-09T16:14:33.487051204Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-09T16:14:33.710561706Z Collecting performance related data for node ip-10-0-128-73.ec2.internal
2026-06-09T16:14:33.717942727Z Collecting performance related data for node ip-10-0-129-94.ec2.internal
2026-06-09T16:14:33.725497770Z Collecting performance related data for node ip-10-0-136-159.ec2.internal
2026-06-09T16:14:34.992415048Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-09T16:14:36.260630391Z Gathering data for ns/openshift-host-network...
2026-06-09T16:14:36.269782569Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-09T16:14:36.586917276Z Gathering data for ns/openshift-network-diagnostics...
2026-06-09T16:14:37.097742682Z Gathering data for ns/openshift-network-node-identity...
2026-06-09T16:14:37.473271155Z Gathering data for ns/openshift-network-operator...
2026-06-09T16:14:37.829840185Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-09T16:14:38.076313152Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-09T16:14:38.541342313Z Gathering data for ns/openshift-apiserver-operator...
2026-06-09T16:14:38.873389684Z Gathering data for ns/openshift-apiserver...
2026-06-09T16:14:39.249099753Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-09T16:14:39.504811120Z Gathering data for ns/openshift-controller-manager...
2026-06-09T16:14:39.760938363Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-09T16:14:40.393521262Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-09T16:14:40.663041736Z Gathering data for ns/openshift-service-ca-operator...
2026-06-09T16:14:40.985777412Z Gathering data for ns/openshift-service-ca...
2026-06-09T16:14:41.275332641Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-09T16:14:41.323146750Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-09T16:14:42.137293513Z Collecting kubelet logs for node ip-10-0-128-73.ec2.internal
2026-06-09T16:14:42.139056878Z Collecting kubelet logs for node ip-10-0-129-94.ec2.internal
2026-06-09T16:14:42.140928733Z Collecting kubelet logs for node ip-10-0-136-159.ec2.internal
2026-06-09T16:14:42.227040933Z Wrote inspect data to must-gather.
2026-06-09T16:14:42.227209573Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T16:14:42.227209573Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-09T16:14:42.481119421Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-jhfgg namespace
2026-06-09T16:14:42.486118469Z INFO: Node performance data collection complete.
2026-06-09T16:14:42.618789514Z error: the server doesn't have a resource type "clusters"
2026-06-09T16:14:42.768032160Z Caches written to disk