2026-09-11T05:51:34.871937906Z [disk usage checker] Started 2026-09-11T05:51:34.875221667Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T05:51:35.068579529Z Gathering data for ns/openshift-cluster-version... 2026-09-11T05:51:35.343602313Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-11T05:51:35.421978688Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-11T05:51:35.496402817Z Gathering data for ns/default... 2026-09-11T05:51:35.810173485Z Gathering data for ns/openshift... 2026-09-11T05:51:36.182602053Z Gathering data for ns/kube-system... 2026-09-11T05:51:36.830841259Z Gathering data for ns/openshift-etcd... 2026-09-11T05:51:37.112325526Z Wrote inspect data to must-gather. 2026-09-11T05:51:37.112355036Z error: inspection completed with the errors occurred while gathering data: 2026-09-11T05:51:37.112355036Z namespaces "assisted-installer" not found 2026-09-11T05:51:37.357854733Z Waiting on subprocesses to finish execution. 2026-09-11T05:51:37.362844126Z INFO: Gathering HAProxy config files 2026-09-11T05:51:37.367223148Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-11T05:51:37.372088157Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-11T05:51:37.381294953Z Executing Istio gather script 2026-09-11T05:51:37.382029430Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-11T05:51:37.393682110Z INFO: Collecting host service logs for crio 2026-09-11T05:51:37.394069299Z INFO: Collecting host service logs for kubelet 2026-09-11T05:51:37.394427389Z INFO: Collecting host service logs for rpm-ostreed 2026-09-11T05:51:37.394795367Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-11T05:51:37.395144776Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-11T05:51:37.395487117Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-11T05:51:37.395858923Z INFO: Collecting host service logs for NetworkManager 2026-09-11T05:51:37.396209409Z INFO: Collecting host service logs for openvswitch 2026-09-11T05:51:37.396601205Z INFO: Collecting host service logs for ovs-configuration 2026-09-11T05:51:37.396955059Z INFO: Collecting host service logs for ovsdb-server 2026-09-11T05:51:37.397298593Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-11T05:51:37.397804876Z INFO: Waiting for worker host service log collection to complete ... 2026-09-11T05:51:37.400419875Z INFO: Waiting for node performance related collection to complete ... 2026-09-11T05:51:37.645495779Z INFO: Collecting Insights Archives from insights-operator-5844b856db-v6wtq 2026-09-11T05:51:37.645573118Z insights-runtime-extractor-5bzvr 2026-09-11T05:51:37.645598169Z insights-runtime-extractor-rxb5g 2026-09-11T05:51:37.645617554Z insights-runtime-extractor-vch8h 2026-09-11T05:51:37.878565927Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-11T05:51:37.923363415Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-11T05:51:38.066279862Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-11T05:51:38.066352975Z INFO: on-disk MachineConfig config collection complete. 2026-09-11T05:51:38.251704876Z No resources found in openshift-etcd namespace. 2026-09-11T05:51:38.292388698Z Wrote inspect data to must-gather. 2026-09-11T05:51:38.399159998Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-11T05:51:38.431532051Z No resources found 2026-09-11T05:51:38.442334301Z error: the server doesn't have a resource type "performanceprofile" 2026-09-11T05:51:38.584301507Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-11T05:51:38.584301507Z See 'oc rsync -h' for help and examples 2026-09-11T05:51:38.698463999Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-11T05:51:38.711071573Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-11T05:51:38.801253296Z INFO: Worker host service log collection to complete. 2026-09-11T05:51:38.853875532Z ERROR: No running kube-apiserver pods found 2026-09-11T05:51:38.902756830Z Wrote inspect data to must-gather. 2026-09-11T05:51:38.920938185Z INFO: "metallb-operator" not detected. Skipping. 2026-09-11T05:51:39.032002302Z INFO: 'previous-logs' folder not found on ip-10-0-133-194.ec2.internal, skipping... 2026-09-11T05:51:39.062148132Z Wrote inspect data to must-gather. 2026-09-11T05:51:39.316614768Z Gathering data for ns/openshift-monitoring... 2026-09-11T05:51:39.428653710Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-11T05:51:39.444078827Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-11T05:51:39.555357620Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-11T05:51:39.651614785Z No resources found 2026-09-11T05:51:39.662528838Z INFO: Getting rules from prometheus-k8s-0 2026-09-11T05:51:39.771383762Z Wrote inspect data to must-gather. 2026-09-11T05:51:39.788185846Z error: the server doesn't have a resource type "machineconfigs" 2026-09-11T05:51:39.830331999Z Wrote inspect data to must-gather. 2026-09-11T05:51:39.905015393Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T05:51:39.957325381Z Wrote inspect data to must-gather. 2026-09-11T05:51:39.963949705Z INFO: 'previous-logs' folder not found on ip-10-0-137-187.ec2.internal, skipping... 2026-09-11T05:51:39.998099075Z Wrote inspect data to must-gather. 2026-09-11T05:51:40.018241723Z tar: Removing leading `/' from member names 2026-09-11T05:51:40.023659215Z INFO: HAProxy config collection complete. 2026-09-11T05:51:40.071369051Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-11T05:51:40.278201293Z INFO: 'previous-logs' folder not found on ip-10-0-138-238.ec2.internal, skipping... 2026-09-11T05:51:40.279530661Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-11T05:51:40.280316522Z INFO: Machine Config Daemon termination log collection complete. 2026-09-11T05:51:40.288537871Z INFO: Getting status/config from prometheus-k8s-0 2026-09-11T05:51:40.291217140Z Wrote inspect data to must-gather. 2026-09-11T05:51:40.316583031Z Wrote inspect data to must-gather. 2026-09-11T05:51:40.469748554Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-11T05:51:40.527900444Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-11T05:51:40.536673195Z Wrote inspect data to must-gather. 2026-09-11T05:51:40.663761951Z Wrote inspect data to must-gather. 2026-09-11T05:51:40.725511396Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-11T05:51:40.811005325Z Wrote inspect data to must-gather. 2026-09-11T05:51:40.901410456Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-11T05:51:40.904272327Z Wrote inspect data to must-gather. 2026-09-11T05:51:40.973458170Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-11T05:51:41.052802269Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-11T05:51:41.093667839Z INFO: INTERCONNECT MODE 2026-09-11T05:51:41.093776041Z INFO: Gathering ovn-kubernetes DBs 2026-09-11T05:51:41.135959398Z Wrote inspect data to must-gather. 2026-09-11T05:51:41.223619038Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-11T05:51:41.265798571Z INFO: Gathering OVN_Northbound from ovnkube-node-78lkv... 2026-09-11T05:51:41.266080150Z INFO: Gathering OVN_Northbound from ovnkube-node-gr7zt... 2026-09-11T05:51:41.266398380Z INFO: Gathering OVN_Northbound from ovnkube-node-srxzk... 2026-09-11T05:51:41.266855557Z INFO: Gathering OVN_Southbound from ovnkube-node-78lkv... 2026-09-11T05:51:41.267292348Z INFO: Gathering OVN_Southbound from ovnkube-node-gr7zt... 2026-09-11T05:51:41.267670498Z INFO: Gathering OVN_Southbound from ovnkube-node-srxzk... 2026-09-11T05:51:41.271059954Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:41.487564391Z Wrote inspect data to must-gather. 2026-09-11T05:51:41.565049628Z tar: Removing leading `/' from member names 2026-09-11T05:51:41.600384244Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-11T05:51:41.613560543Z tar: Removing leading `/' from member names 2026-09-11T05:51:41.649694184Z tar: Removing leading `/' from member names 2026-09-11T05:51:41.649823998Z tar: Removing leading `/' from member names 2026-09-11T05:51:41.659897616Z tar: Removing leading `/' from member names 2026-09-11T05:51:41.679199497Z tar: Removing leading `/' from member names 2026-09-11T05:51:41.716517036Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:41.720087225Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-11T05:51:41.723898528Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-11T05:51:41.914394397Z Wrote inspect data to must-gather. 2026-09-11T05:51:41.965818117Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:41.968906253Z Gathering data for ns/openshift-network-console... 2026-09-11T05:51:42.082722799Z INFO: Getting status from alertmanager-main-0 2026-09-11T05:51:42.199502807Z INFO: Waiting for network log collection to complete ... 2026-09-11T05:51:42.199561212Z INFO: Waiting for ovnk database copies to complete ... 2026-09-11T05:51:42.200079540Z INFO: Copying ovnk databases complete. 2026-09-11T05:51:42.201945735Z 3.6M must-gather/network_logs/ovnk_database_store 2026-09-11T05:51:42.205129429Z ovnk_database_store/ 2026-09-11T05:51:42.205936324Z ovnk_database_store/ovnkube-node-gr7zt_nbdb 2026-09-11T05:51:42.218620386Z ovnk_database_store/ovnkube-node-gr7zt_sbdb 2026-09-11T05:51:42.225684041Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:42.229944645Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-11T05:51:42.242602923Z ovnk_database_store/ovnkube-node-srxzk_nbdb 2026-09-11T05:51:42.251595454Z ovnk_database_store/ovnkube-node-srxzk_sbdb 2026-09-11T05:51:42.276818328Z ovnk_database_store/ovnkube-node-78lkv_sbdb 2026-09-11T05:51:42.277307414Z W0911 05:51:42.277276 1478 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-11T05:51:42.293210131Z Wrote inspect data to must-gather. 2026-09-11T05:51:42.301729903Z ovnk_database_store/ovnkube-node-78lkv_nbdb 2026-09-11T05:51:42.312749517Z INFO: Network log collection complete. 2026-09-11T05:51:42.447363656Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:42.472623489Z Gathering data for ns/openshift-console-operator... 2026-09-11T05:51:42.587632122Z Wrote inspect data to must-gather. 2026-09-11T05:51:42.637350135Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:42.639666379Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-11T05:51:42.832338550Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-11T05:51:42.834808371Z INFO: Fallback to identify the container image from release info 2026-09-11T05:51:42.862693732Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:42.872085010Z Gathering data for ns/openshift-console... 2026-09-11T05:51:43.054482121Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:43.056746204Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-11T05:51:43.194300922Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:43.390510739Z Wrote inspect data to /must-gather/istio. 2026-09-11T05:51:43.410421935Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-11T05:51:43.503869177Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-11T05:51:43.627774020Z daemonset.apps/perf-node-gather-daemonset created 2026-09-11T05:51:43.771234070Z Waiting for performance profile collector pods to become ready: 1 2026-09-11T05:51:43.952161047Z Gathering data for ns/openshift-dns-operator... 2026-09-11T05:51:44.096944302Z Done executing Istio gather script 2026-09-11T05:51:44.232025226Z Gathering data for ns/openshift-dns... 2026-09-11T05:51:44.829593569Z Gathering data for ns/openshift-image-registry... 2026-09-11T05:51:44.882311190Z Waiting for performance profile collector pods to become ready: 2 2026-09-11T05:51:44.916598172Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T05:51:45.274192844Z Gathering data for ns/openshift-ingress-operator... 2026-09-11T05:51:45.566117923Z Gathering data for ns/openshift-ingress... 2026-09-11T05:51:45.878926786Z Gathering data for ns/openshift-ingress-canary... 2026-09-11T05:51:45.987872134Z Waiting for performance profile collector pods to become ready: 3 2026-09-11T05:51:46.246505862Z Gathering data for ns/openshift-insights... 2026-09-11T05:51:47.094325691Z Waiting for performance profile collector pods to become ready: 4 2026-09-11T05:51:48.177824666Z Gathering data for ns/openshift-config... 2026-09-11T05:51:48.206151686Z Waiting for performance profile collector pods to become ready: 5 2026-09-11T05:51:48.423347933Z Gathering data for ns/openshift-config-managed... 2026-09-11T05:51:48.806509624Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-11T05:51:49.047134220Z Gathering data for ns/openshift-kube-apiserver... 2026-09-11T05:51:49.288810454Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-11T05:51:49.321633439Z Waiting for performance profile collector pods to become ready: 6 2026-09-11T05:51:49.544735036Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-11T05:51:49.798297994Z Gathering data for ns/openshift-kube-scheduler... 2026-09-11T05:51:49.920943806Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T05:51:50.042019489Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-11T05:51:50.307743423Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-11T05:51:50.439042707Z Waiting for performance profile collector pods to become ready: 7 2026-09-11T05:51:50.617888040Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-11T05:51:50.925356895Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-11T05:51:51.404299114Z Gathering data for ns/openshift-multus... 2026-09-11T05:51:51.538743152Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-11T05:51:51.749637488Z Collecting performance related data for node ip-10-0-137-187.ec2.internal 2026-09-11T05:51:51.756154740Z Collecting performance related data for node ip-10-0-133-194.ec2.internal 2026-09-11T05:51:51.763123886Z Collecting performance related data for node ip-10-0-138-238.ec2.internal 2026-09-11T05:51:52.878246619Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-11T05:51:54.104904187Z Gathering data for ns/openshift-host-network... 2026-09-11T05:51:54.416352590Z Gathering data for ns/openshift-network-diagnostics... 2026-09-11T05:51:54.925843909Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T05:51:54.930304076Z Gathering data for ns/openshift-network-node-identity... 2026-09-11T05:51:55.319469349Z Gathering data for ns/openshift-network-operator... 2026-09-11T05:51:55.683955402Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-11T05:51:55.942418272Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-11T05:51:56.407985448Z Gathering data for ns/openshift-apiserver-operator... 2026-09-11T05:51:56.693602778Z Gathering data for ns/openshift-apiserver... 2026-09-11T05:51:57.069233397Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-11T05:51:57.312310173Z Gathering data for ns/openshift-controller-manager... 2026-09-11T05:51:57.565657285Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-11T05:51:58.178631348Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-11T05:51:58.444192722Z Gathering data for ns/openshift-service-ca-operator... 2026-09-11T05:51:58.734593207Z Gathering data for ns/openshift-service-ca... 2026-09-11T05:51:59.073842725Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-11T05:51:59.348525026Z Collecting kubelet logs for node ip-10-0-137-187.ec2.internal 2026-09-11T05:51:59.350301626Z Collecting kubelet logs for node ip-10-0-133-194.ec2.internal 2026-09-11T05:51:59.351919390Z Collecting kubelet logs for node ip-10-0-138-238.ec2.internal 2026-09-11T05:51:59.705311431Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-gbwzt namespace 2026-09-11T05:51:59.711170308Z INFO: Node performance data collection complete. 2026-09-11T05:51:59.931254648Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-11T05:52:00.107196316Z Wrote inspect data to must-gather. 2026-09-11T05:52:00.107232531Z error: inspection completed with the errors occurred while gathering data: 2026-09-11T05:52:00.107232531Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-11T05:52:00.224911124Z error: the server doesn't have a resource type "clusters" 2026-09-11T05:52:00.347953922Z Caches written to disk