2026-07-14T03:34:45.366336787Z [disk usage checker] Started 2026-07-14T03:34:45.369636301Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T03:34:45.555579067Z Gathering data for ns/openshift-cluster-version... 2026-07-14T03:34:45.679094164Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-14T03:34:45.754019503Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-14T03:34:45.827708858Z Gathering data for ns/default... 2026-07-14T03:34:46.100292695Z Gathering data for ns/openshift... 2026-07-14T03:34:46.480079474Z Gathering data for ns/kube-system... 2026-07-14T03:34:47.155390032Z Gathering data for ns/openshift-etcd... 2026-07-14T03:34:47.426416842Z Wrote inspect data to must-gather. 2026-07-14T03:34:47.426455671Z error: inspection completed with the errors occurred while gathering data: 2026-07-14T03:34:47.426455671Z namespaces "assisted-installer" not found 2026-07-14T03:34:47.799846322Z Waiting on subprocesses to finish execution. 2026-07-14T03:34:47.808003157Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-14T03:34:47.811040798Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-14T03:34:47.812378361Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-14T03:34:47.813227604Z Executing Istio gather script 2026-07-14T03:34:47.813583197Z INFO: Gathering HAProxy config files 2026-07-14T03:34:47.826973333Z INFO: Collecting host service logs for crio 2026-07-14T03:34:47.827312644Z INFO: Collecting host service logs for kubelet 2026-07-14T03:34:47.827666139Z INFO: Collecting host service logs for rpm-ostreed 2026-07-14T03:34:47.828015731Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-14T03:34:47.828339648Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-14T03:34:47.828665309Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-14T03:34:47.829041596Z INFO: Collecting host service logs for NetworkManager 2026-07-14T03:34:47.829408924Z INFO: Collecting host service logs for openvswitch 2026-07-14T03:34:47.829753436Z INFO: Collecting host service logs for ovs-configuration 2026-07-14T03:34:47.830609276Z INFO: Collecting host service logs for ovsdb-server 2026-07-14T03:34:47.830980668Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-14T03:34:47.831368211Z INFO: Waiting for worker host service log collection to complete ... 2026-07-14T03:34:48.227392077Z No resources found in openshift-etcd namespace. 2026-07-14T03:34:48.297073047Z INFO: Waiting for node performance related collection to complete ... 2026-07-14T03:34:48.480619429Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-14T03:34:48.608038589Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-14T03:34:48.693718042Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-14T03:34:48.693834344Z INFO: on-disk MachineConfig config collection complete. 2026-07-14T03:34:48.735070464Z INFO: "metallb-operator" not detected. Skipping. 2026-07-14T03:34:48.823742840Z Wrote inspect data to must-gather. 2026-07-14T03:34:48.953923614Z error: the server doesn't have a resource type "performanceprofile" 2026-07-14T03:34:49.052066766Z ERROR: No running kube-apiserver pods found 2026-07-14T03:34:49.140600696Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-14T03:34:49.149460053Z INFO: Worker host service log collection to complete. 2026-07-14T03:34:49.199500507Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-14T03:34:49.202564180Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-mxx2f 2026-07-14T03:34:49.202564180Z insights-runtime-extractor-2xqdl 2026-07-14T03:34:49.202564180Z insights-runtime-extractor-9ddh9 2026-07-14T03:34:49.202564180Z insights-runtime-extractor-ngbh5 2026-07-14T03:34:49.255136706Z No resources found 2026-07-14T03:34:49.284321209Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-14T03:34:49.407402713Z INFO: 'previous-logs' folder not found on ip-10-0-130-194.ec2.internal, skipping... 2026-07-14T03:34:49.455610099Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-14T03:34:49.763405724Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-14T03:34:49.763405724Z See 'oc rsync -h' for help and examples 2026-07-14T03:34:49.774137942Z Gathering data for ns/openshift-monitoring... 2026-07-14T03:34:49.802314599Z No resources found 2026-07-14T03:34:49.921189348Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-14T03:34:49.950445998Z Wrote inspect data to must-gather. 2026-07-14T03:34:49.994155137Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.080018797Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.155628269Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-14T03:34:50.262827454Z tar: Removing leading `/' from member names 2026-07-14T03:34:50.271055578Z INFO: HAProxy config collection complete. 2026-07-14T03:34:50.321285485Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.339687236Z error: the server doesn't have a resource type "machineconfigs" 2026-07-14T03:34:50.340438087Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.365647578Z INFO: 'previous-logs' folder not found on ip-10-0-136-173.ec2.internal, skipping... 2026-07-14T03:34:50.378133850Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T03:34:50.378462458Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.493394075Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-14T03:34:50.532593080Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-14T03:34:50.579167560Z INFO: Getting rules from prometheus-k8s-0 2026-07-14T03:34:50.620922020Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.655106863Z INFO: 'previous-logs' folder not found on ip-10-0-140-96.ec2.internal, skipping... 2026-07-14T03:34:50.655161594Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-14T03:34:50.655161594Z INFO: Machine Config Daemon termination log collection complete. 2026-07-14T03:34:50.727617490Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.853771261Z Wrote inspect data to must-gather. 2026-07-14T03:34:50.857243447Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-14T03:34:50.976908371Z INFO: Getting status/config from prometheus-k8s-0 2026-07-14T03:34:51.082014308Z Wrote inspect data to must-gather. 2026-07-14T03:34:51.103381237Z Wrote inspect data to must-gather. 2026-07-14T03:34:51.277232340Z Wrote inspect data to must-gather. 2026-07-14T03:34:51.356981407Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-14T03:34:51.369305827Z INFO: INTERCONNECT MODE 2026-07-14T03:34:51.369340396Z INFO: Gathering ovn-kubernetes DBs 2026-07-14T03:34:51.399918380Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-14T03:34:51.502978797Z Wrote inspect data to must-gather. 2026-07-14T03:34:51.551002568Z INFO: Gathering OVN_Northbound from ovnkube-node-95jxw... 2026-07-14T03:34:51.551312297Z INFO: Gathering OVN_Northbound from ovnkube-node-kgcqq... 2026-07-14T03:34:51.551691963Z INFO: Gathering OVN_Northbound from ovnkube-node-t4ff7... 2026-07-14T03:34:51.552007430Z INFO: Gathering OVN_Southbound from ovnkube-node-95jxw... 2026-07-14T03:34:51.552264384Z INFO: Gathering OVN_Southbound from ovnkube-node-kgcqq... 2026-07-14T03:34:51.552535278Z INFO: Gathering OVN_Southbound from ovnkube-node-t4ff7... 2026-07-14T03:34:51.632251008Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-14T03:34:51.657316862Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:51.698004172Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-14T03:34:51.842766265Z tar: Removing leading `/' from member names 2026-07-14T03:34:51.843012058Z tar: Removing leading `/' from member names 2026-07-14T03:34:51.849979873Z tar: Removing leading `/' from member names 2026-07-14T03:34:51.852485167Z tar: Removing leading `/' from member names 2026-07-14T03:34:51.876237429Z Wrote inspect data to must-gather. 2026-07-14T03:34:51.881240715Z tar: Removing leading `/' from member names 2026-07-14T03:34:51.897188568Z tar: Removing leading `/' from member names 2026-07-14T03:34:51.985005282Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:51.988400477Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-14T03:34:51.997353458Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-14T03:34:52.111645795Z Wrote inspect data to must-gather. 2026-07-14T03:34:52.230564515Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:52.292411450Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-14T03:34:52.343536887Z W0714 03:34:52.343498 1473 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-14T03:34:52.348770209Z Wrote inspect data to must-gather. 2026-07-14T03:34:52.376392788Z INFO: Waiting for network log collection to complete ... 2026-07-14T03:34:52.376627058Z INFO: Waiting for ovnk database copies to complete ... 2026-07-14T03:34:52.377172036Z INFO: Copying ovnk databases complete. 2026-07-14T03:34:52.379252149Z 2.5M must-gather/network_logs/ovnk_database_store 2026-07-14T03:34:52.382075336Z ovnk_database_store/ 2026-07-14T03:34:52.382124498Z ovnk_database_store/ovnkube-node-kgcqq_nbdb 2026-07-14T03:34:52.389821777Z ovnk_database_store/ovnkube-node-kgcqq_sbdb 2026-07-14T03:34:52.392452275Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-14T03:34:52.406258069Z ovnk_database_store/ovnkube-node-95jxw_nbdb 2026-07-14T03:34:52.413878090Z ovnk_database_store/ovnkube-node-t4ff7_sbdb 2026-07-14T03:34:52.432833891Z ovnk_database_store/ovnkube-node-t4ff7_nbdb 2026-07-14T03:34:52.440417066Z ovnk_database_store/ovnkube-node-95jxw_sbdb 2026-07-14T03:34:52.460902143Z INFO: Network log collection complete. 2026-07-14T03:34:52.470441033Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:52.473406598Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-14T03:34:52.590216794Z Gathering data for ns/openshift-network-console... 2026-07-14T03:34:52.657152280Z Wrote inspect data to must-gather. 2026-07-14T03:34:52.673042822Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:52.697611352Z INFO: Getting status from alertmanager-main-0 2026-07-14T03:34:52.874646012Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:52.878063767Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-14T03:34:52.918542299Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-14T03:34:52.921278028Z INFO: Fallback to identify the container image from release info 2026-07-14T03:34:53.096651871Z Gathering data for ns/openshift-console-operator... 2026-07-14T03:34:53.102561297Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:53.280079140Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:53.282040801Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-14T03:34:53.431290321Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:53.511140640Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d 2026-07-14T03:34:53.531156295Z Gathering data for ns/openshift-console... 2026-07-14T03:34:53.624678676Z Wrote inspect data to /must-gather/istio. 2026-07-14T03:34:53.635002060Z daemonset.apps/perf-node-gather-daemonset created 2026-07-14T03:34:53.763455502Z Waiting for performance profile collector pods to become ready: 1 2026-07-14T03:34:54.161354816Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-14T03:34:54.304282648Z Done executing Istio gather script 2026-07-14T03:34:54.680309377Z Gathering data for ns/openshift-dns-operator... 2026-07-14T03:34:54.864642447Z Waiting for performance profile collector pods to become ready: 2 2026-07-14T03:34:54.968245277Z Gathering data for ns/openshift-dns... 2026-07-14T03:34:55.383028300Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T03:34:55.588505047Z Gathering data for ns/openshift-image-registry... 2026-07-14T03:34:55.973822363Z Waiting for performance profile collector pods to become ready: 3 2026-07-14T03:34:56.043588393Z Gathering data for ns/openshift-ingress-operator... 2026-07-14T03:34:56.356664975Z Gathering data for ns/openshift-ingress... 2026-07-14T03:34:56.653014591Z Gathering data for ns/openshift-ingress-canary... 2026-07-14T03:34:57.032577770Z Gathering data for ns/openshift-insights... 2026-07-14T03:34:57.081168748Z Waiting for performance profile collector pods to become ready: 4 2026-07-14T03:34:58.185986696Z Waiting for performance profile collector pods to become ready: 5 2026-07-14T03:34:58.773041073Z Gathering data for ns/openshift-config... 2026-07-14T03:34:59.018318579Z Gathering data for ns/openshift-config-managed... 2026-07-14T03:34:59.298743658Z Waiting for performance profile collector pods to become ready: 6 2026-07-14T03:34:59.420235649Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-14T03:34:59.651943628Z Gathering data for ns/openshift-kube-apiserver... 2026-07-14T03:34:59.881362074Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-14T03:35:00.118651530Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-14T03:35:00.387747130Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T03:35:00.419150684Z Waiting for performance profile collector pods to become ready: 7 2026-07-14T03:35:00.433312822Z Gathering data for ns/openshift-kube-scheduler... 2026-07-14T03:35:00.666317008Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-14T03:35:00.926394246Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-14T03:35:01.214153869Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-14T03:35:01.504875663Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-14T03:35:01.543429173Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-14T03:35:01.781082872Z Collecting performance related data for node ip-10-0-130-194.ec2.internal 2026-07-14T03:35:01.790669854Z Collecting performance related data for node ip-10-0-136-173.ec2.internal 2026-07-14T03:35:01.801017666Z Collecting performance related data for node ip-10-0-140-96.ec2.internal 2026-07-14T03:35:02.121765012Z Gathering data for ns/openshift-multus... 2026-07-14T03:35:03.907690643Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-14T03:35:05.266545967Z Gathering data for ns/openshift-host-network... 2026-07-14T03:35:05.392931041Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T03:35:05.563503294Z Gathering data for ns/openshift-network-diagnostics... 2026-07-14T03:35:06.093259444Z Gathering data for ns/openshift-network-node-identity... 2026-07-14T03:35:06.517196933Z Gathering data for ns/openshift-network-operator... 2026-07-14T03:35:06.903442292Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-14T03:35:07.161693683Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-14T03:35:07.604008314Z Gathering data for ns/openshift-apiserver-operator... 2026-07-14T03:35:07.868990984Z Gathering data for ns/openshift-apiserver... 2026-07-14T03:35:08.209852713Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-14T03:35:08.436188782Z Gathering data for ns/openshift-controller-manager... 2026-07-14T03:35:08.672338822Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-14T03:35:09.295346347Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-14T03:35:09.596577900Z Gathering data for ns/openshift-service-ca-operator... 2026-07-14T03:35:09.741971030Z Collecting kubelet logs for node ip-10-0-130-194.ec2.internal 2026-07-14T03:35:09.743867332Z Collecting kubelet logs for node ip-10-0-136-173.ec2.internal 2026-07-14T03:35:09.745638550Z Collecting kubelet logs for node ip-10-0-140-96.ec2.internal 2026-07-14T03:35:09.929921534Z Gathering data for ns/openshift-service-ca... 2026-07-14T03:35:10.078590563Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-97jd8 namespace 2026-07-14T03:35:10.083675315Z INFO: Node performance data collection complete. 2026-07-14T03:35:10.273015563Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-14T03:35:10.397377993Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T03:35:11.117810290Z Wrote inspect data to must-gather. 2026-07-14T03:35:11.117855230Z error: inspection completed with the errors occurred while gathering data: 2026-07-14T03:35:11.117855230Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-14T03:35:11.238099457Z error: the server doesn't have a resource type "clusters" 2026-07-14T03:35:11.338269717Z Caches written to disk