2026-09-08T11:11:06.073422686Z [disk usage checker] Started
2026-09-08T11:11:06.077359453Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-09-08T11:11:06.266517904Z Gathering data for ns/openshift-cluster-version...
2026-09-08T11:11:06.400179357Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-08T11:11:06.485207334Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-08T11:11:06.571520763Z Gathering data for ns/default...
2026-09-08T11:11:06.862866152Z Gathering data for ns/openshift...
2026-09-08T11:11:07.270651495Z Gathering data for ns/kube-system...
2026-09-08T11:11:07.972751565Z Gathering data for ns/openshift-etcd...
2026-09-08T11:11:08.283228016Z Wrote inspect data to must-gather.
2026-09-08T11:11:08.283265425Z error: inspection completed with the errors occurred while gathering data:
2026-09-08T11:11:08.283265425Z     namespaces "assisted-installer" not found
2026-09-08T11:11:08.586317141Z Waiting on subprocesses to finish execution.
2026-09-08T11:11:08.588661242Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-08T11:11:08.595720945Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-08T11:11:08.596680614Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-08T11:11:08.607924054Z INFO: Gathering HAProxy config files
2026-09-08T11:11:08.610250075Z Executing Istio gather script
2026-09-08T11:11:08.627639508Z INFO: Collecting host service logs for crio
2026-09-08T11:11:08.627909256Z INFO: Collecting host service logs for kubelet
2026-09-08T11:11:08.628218606Z INFO: Collecting host service logs for rpm-ostreed
2026-09-08T11:11:08.628517789Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-08T11:11:08.628797982Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-08T11:11:08.629122008Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-08T11:11:08.629502304Z INFO: Collecting host service logs for NetworkManager
2026-09-08T11:11:08.629831594Z INFO: Collecting host service logs for openvswitch
2026-09-08T11:11:08.630185722Z INFO: Collecting host service logs for ovs-configuration
2026-09-08T11:11:08.630565447Z INFO: Collecting host service logs for ovsdb-server
2026-09-08T11:11:08.630935554Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-08T11:11:08.631377180Z INFO: Waiting for worker host service log collection to complete ...
2026-09-08T11:11:08.755448819Z INFO: Waiting for node performance related collection to complete ...
2026-09-08T11:11:09.077581126Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-lnsqs
2026-09-08T11:11:09.077581126Z insights-runtime-extractor-czgb6
2026-09-08T11:11:09.077581126Z insights-runtime-extractor-fb9h9
2026-09-08T11:11:09.077581126Z insights-runtime-extractor-qvz6z
2026-09-08T11:11:09.116291215Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-08T11:11:09.116390479Z INFO: on-disk MachineConfig config collection complete.
2026-09-08T11:11:09.172655494Z No resources found in openshift-etcd namespace.
2026-09-08T11:11:09.397160287Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-08T11:11:09.400133373Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-08T11:11:09.532284532Z INFO: "metallb-operator" not detected. Skipping.
2026-09-08T11:11:09.543903650Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-08T11:11:09.795806455Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-08T11:11:09.958948972Z Wrote inspect data to must-gather.
2026-09-08T11:11:09.961735485Z No resources found
2026-09-08T11:11:10.061794364Z error: the server doesn't have a resource type "performanceprofile"
2026-09-08T11:11:10.141742703Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-08T11:11:10.141742703Z See 'oc rsync -h' for help and examples
2026-09-08T11:11:10.200628721Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-08T11:11:10.268670997Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-08T11:11:10.361075696Z ERROR: No running kube-apiserver pods found
2026-09-08T11:11:10.503962140Z INFO: Worker host service log collection to complete.
2026-09-08T11:11:10.653183137Z Wrote inspect data to must-gather.
2026-09-08T11:11:10.964356384Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.008655538Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-08T11:11:11.016590277Z No resources found
2026-09-08T11:11:11.027523290Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.031780980Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.037568383Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-08T11:11:11.041055562Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.112668907Z INFO: 'previous-logs' folder not found on ip-10-0-128-61.ec2.internal, skipping...
2026-09-08T11:11:11.121706831Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-09-08T11:11:11.158170026Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.175111491Z tar: Removing leading `/' from member names
2026-09-08T11:11:11.179026682Z INFO: HAProxy config collection complete.
2026-09-08T11:11:11.208818931Z Gathering data for ns/openshift-monitoring...
2026-09-08T11:11:11.297719928Z error: the server doesn't have a resource type "machineconfigs"
2026-09-08T11:11:11.365450423Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.368175336Z INFO: Getting rules from prometheus-k8s-0
2026-09-08T11:11:11.415494526Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-08T11:11:11.455327163Z INFO: 'previous-logs' folder not found on ip-10-0-130-163.ec2.internal, skipping...
2026-09-08T11:11:11.481636216Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-08T11:11:11.618759868Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.701201674Z Wrote inspect data to must-gather.
2026-09-08T11:11:11.755672050Z INFO: 'previous-logs' folder not found on ip-10-0-136-160.ec2.internal, skipping...
2026-09-08T11:11:11.755802972Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-08T11:11:11.755865114Z INFO: Machine Config Daemon termination log collection complete.
2026-09-08T11:11:11.792141415Z INFO: Getting status/config from prometheus-k8s-0
2026-09-08T11:11:11.863407714Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-08T11:11:11.875362696Z Wrote inspect data to must-gather.
2026-09-08T11:11:12.067847001Z Wrote inspect data to must-gather.
2026-09-08T11:11:12.165442780Z INFO: INTERCONNECT MODE
2026-09-08T11:11:12.165474106Z INFO: Gathering ovn-kubernetes DBs
2026-09-08T11:11:12.193773099Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-08T11:11:12.284330963Z Wrote inspect data to must-gather.
2026-09-08T11:11:12.329525964Z INFO: Gathering OVN_Northbound from ovnkube-node-cxcww...
2026-09-08T11:11:12.329806567Z INFO: Gathering OVN_Northbound from ovnkube-node-hrhjv...
2026-09-08T11:11:12.330154319Z INFO: Gathering OVN_Northbound from ovnkube-node-pkkzj...
2026-09-08T11:11:12.330448797Z INFO: Gathering OVN_Southbound from ovnkube-node-cxcww...
2026-09-08T11:11:12.330745003Z INFO: Gathering OVN_Southbound from ovnkube-node-hrhjv...
2026-09-08T11:11:12.331238063Z INFO: Gathering OVN_Southbound from ovnkube-node-pkkzj...
2026-09-08T11:11:12.432802238Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-08T11:11:12.542762167Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-08T11:11:12.622710973Z Wrote inspect data to must-gather.
2026-09-08T11:11:12.624417794Z tar: Removing leading `/' from member names
2026-09-08T11:11:12.676984824Z tar: Removing leading `/' from member names
2026-09-08T11:11:12.686709858Z tar: Removing leading `/' from member names
2026-09-08T11:11:12.699181759Z tar: Removing leading `/' from member names
2026-09-08T11:11:12.713096267Z tar: Removing leading `/' from member names
2026-09-08T11:11:12.749446913Z tar: Removing leading `/' from member names
2026-09-08T11:11:12.808089881Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:12.818193023Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-08T11:11:12.826576408Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-08T11:11:12.884977762Z Wrote inspect data to must-gather.
2026-09-08T11:11:13.071178754Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:13.073624911Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-08T11:11:13.102724546Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-08T11:11:13.144504923Z Wrote inspect data to must-gather.
2026-09-08T11:11:13.248784586Z INFO: Waiting for network log collection to complete ...
2026-09-08T11:11:13.248822330Z INFO: Waiting for ovnk database copies to complete ...
2026-09-08T11:11:13.249365513Z INFO: Copying ovnk databases complete.
2026-09-08T11:11:13.251078902Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-08T11:11:13.253682229Z ovnk_database_store/
2026-09-08T11:11:13.253682229Z ovnk_database_store/ovnkube-node-hrhjv_sbdb
2026-09-08T11:11:13.273516663Z ovnk_database_store/ovnkube-node-cxcww_sbdb
2026-09-08T11:11:13.293392621Z ovnk_database_store/ovnkube-node-pkkzj_sbdb
2026-09-08T11:11:13.298231149Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:13.313071344Z ovnk_database_store/ovnkube-node-cxcww_nbdb
2026-09-08T11:11:13.319816886Z ovnk_database_store/ovnkube-node-hrhjv_nbdb
2026-09-08T11:11:13.328156067Z ovnk_database_store/ovnkube-node-pkkzj_nbdb
2026-09-08T11:11:13.339403757Z INFO: Network log collection complete.
2026-09-08T11:11:13.379698132Z W0908 11:11:13.379656    1509 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-08T11:11:13.385645509Z Wrote inspect data to must-gather.
2026-09-08T11:11:13.501488618Z INFO: Getting status from alertmanager-main-0
2026-09-08T11:11:13.505227579Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-08T11:11:13.508597380Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:13.512475346Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-08T11:11:13.690072213Z Wrote inspect data to must-gather.
2026-09-08T11:11:13.709707855Z Gathering data for ns/openshift-network-console...
2026-09-08T11:11:13.722674953Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:13.935965589Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:13.938509681Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-08T11:11:13.952237122Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-08T11:11:13.955599848Z INFO: Fallback to identify the container image from release info
2026-09-08T11:11:14.165661356Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:14.171594779Z Gathering data for ns/openshift-console-operator...
2026-09-08T11:11:14.336571940Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:14.338598783Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-08T11:11:14.455034947Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-08T11:11:14.516248332Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:14.578656229Z Gathering data for ns/openshift-console...
2026-09-08T11:11:14.589556843Z daemonset.apps/perf-node-gather-daemonset created
2026-09-08T11:11:14.721684925Z Wrote inspect data to /must-gather/istio.
2026-09-08T11:11:14.724096868Z Waiting for performance profile collector pods to become ready: 1
2026-09-08T11:11:15.180225278Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-08T11:11:15.375134510Z Done executing Istio gather script
2026-09-08T11:11:15.671269231Z Gathering data for ns/openshift-dns-operator...
2026-09-08T11:11:15.826499362Z Waiting for performance profile collector pods to become ready: 2
2026-09-08T11:11:15.945120726Z Gathering data for ns/openshift-dns...
2026-09-08T11:11:16.127135514Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-09-08T11:11:16.528659290Z Gathering data for ns/openshift-image-registry...
2026-09-08T11:11:16.932855298Z Waiting for performance profile collector pods to become ready: 3
2026-09-08T11:11:17.074020104Z Gathering data for ns/openshift-ingress-operator...
2026-09-08T11:11:17.447412365Z Gathering data for ns/openshift-ingress...
2026-09-08T11:11:17.778472088Z Gathering data for ns/openshift-ingress-canary...
2026-09-08T11:11:18.041467407Z Waiting for performance profile collector pods to become ready: 4
2026-09-08T11:11:18.169233677Z Gathering data for ns/openshift-insights...
2026-09-08T11:11:19.136620745Z Waiting for performance profile collector pods to become ready: 5
2026-09-08T11:11:19.952298745Z Gathering data for ns/openshift-jobset-operator...
2026-09-08T11:11:20.253920389Z Waiting for performance profile collector pods to become ready: 6
2026-09-08T11:11:20.728161952Z Gathering data for ns/openshift-config...
2026-09-08T11:11:20.976810727Z Gathering data for ns/openshift-config-managed...
2026-09-08T11:11:21.131900023Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-09-08T11:11:21.373714081Z Waiting for performance profile collector pods to become ready: 7
2026-09-08T11:11:21.391595081Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-08T11:11:21.674523297Z Gathering data for ns/openshift-kube-apiserver...
2026-09-08T11:11:21.911147428Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-08T11:11:22.175400648Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-08T11:11:22.497883465Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-08T11:11:22.503756469Z Gathering data for ns/openshift-kube-scheduler...
2026-09-08T11:11:22.732318640Z Collecting performance related data for node ip-10-0-136-160.ec2.internal
2026-09-08T11:11:22.741652277Z Collecting performance related data for node ip-10-0-128-61.ec2.internal
2026-09-08T11:11:22.751418622Z Collecting performance related data for node ip-10-0-130-163.ec2.internal
2026-09-08T11:11:22.881608391Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-08T11:11:23.218587934Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-08T11:11:23.628375230Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-08T11:11:24.012619700Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-08T11:11:24.557459637Z Gathering data for ns/openshift-multus...
2026-09-08T11:11:26.011520408Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-08T11:11:26.136636929Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-09-08T11:11:27.276430096Z Gathering data for ns/openshift-host-network...
2026-09-08T11:11:27.638069063Z Gathering data for ns/openshift-network-diagnostics...
2026-09-08T11:11:28.163150681Z Gathering data for ns/openshift-network-node-identity...
2026-09-08T11:11:28.613091020Z Gathering data for ns/openshift-network-operator...
2026-09-08T11:11:28.980811628Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-08T11:11:29.229305873Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-08T11:11:29.724111518Z Gathering data for ns/openshift-apiserver-operator...
2026-09-08T11:11:30.037875491Z Gathering data for ns/openshift-apiserver...
2026-09-08T11:11:30.485621768Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-08T11:11:30.617211628Z Collecting kubelet logs for node ip-10-0-136-160.ec2.internal
2026-09-08T11:11:30.618956922Z Collecting kubelet logs for node ip-10-0-128-61.ec2.internal
2026-09-08T11:11:30.620877640Z Collecting kubelet logs for node ip-10-0-130-163.ec2.internal
2026-09-08T11:11:30.818016229Z Gathering data for ns/openshift-controller-manager...
2026-09-08T11:11:30.959394904Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-bb7st namespace
2026-09-08T11:11:30.964400387Z INFO: Node performance data collection complete.
2026-09-08T11:11:31.141355215Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-09-08T11:11:31.162732160Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-08T11:11:31.874293990Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-08T11:11:32.205732258Z Gathering data for ns/openshift-service-ca-operator...
2026-09-08T11:11:32.558947347Z Gathering data for ns/openshift-service-ca...
2026-09-08T11:11:32.932472886Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-08T11:11:33.802016333Z Wrote inspect data to must-gather.
2026-09-08T11:11:33.802051852Z error: inspection completed with the errors occurred while gathering data:
2026-09-08T11:11:33.802051852Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-08T11:11:33.934986754Z error: the server doesn't have a resource type "clusters"
2026-09-08T11:11:34.082279802Z Caches written to disk