2026-09-19T01:15:49.821997177Z [disk usage checker] Started 2026-09-19T01:15:49.825195866Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-19T01:15:50.014945947Z Gathering data for ns/openshift-cluster-version... 2026-09-19T01:15:50.141432775Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-19T01:15:50.229735318Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-19T01:15:50.309977814Z Gathering data for ns/default... 2026-09-19T01:15:50.624798965Z Gathering data for ns/openshift... 2026-09-19T01:15:51.021728081Z Gathering data for ns/kube-system... 2026-09-19T01:15:51.764179135Z Gathering data for ns/openshift-etcd... 2026-09-19T01:15:52.048267673Z Wrote inspect data to must-gather. 2026-09-19T01:15:52.048406845Z error: inspection completed with the errors occurred while gathering data: 2026-09-19T01:15:52.048406845Z namespaces "assisted-installer" not found 2026-09-19T01:15:52.272481509Z Waiting on subprocesses to finish execution. 2026-09-19T01:15:52.281317832Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-19T01:15:52.284139069Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-19T01:15:52.284957070Z Executing Istio gather script 2026-09-19T01:15:52.285948186Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-19T01:15:52.288758544Z INFO: Gathering HAProxy config files 2026-09-19T01:15:52.320970939Z INFO: Collecting host service logs for crio 2026-09-19T01:15:52.321265906Z INFO: Collecting host service logs for kubelet 2026-09-19T01:15:52.321597306Z INFO: Collecting host service logs for rpm-ostreed 2026-09-19T01:15:52.321896828Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-19T01:15:52.322200806Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-19T01:15:52.322525819Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-19T01:15:52.322818285Z INFO: Collecting host service logs for NetworkManager 2026-09-19T01:15:52.323119918Z INFO: Collecting host service logs for openvswitch 2026-09-19T01:15:52.323431147Z INFO: Collecting host service logs for ovs-configuration 2026-09-19T01:15:52.323743795Z INFO: Collecting host service logs for ovsdb-server 2026-09-19T01:15:52.324026021Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-19T01:15:52.324387778Z INFO: Waiting for worker host service log collection to complete ... 2026-09-19T01:15:52.460933084Z INFO: Waiting for node performance related collection to complete ... 2026-09-19T01:15:52.718866849Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-19T01:15:52.718953447Z INFO: on-disk MachineConfig config collection complete. 2026-09-19T01:15:53.020694987Z No resources found in openshift-etcd namespace. 2026-09-19T01:15:53.223777030Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-19T01:15:53.227671650Z Wrote inspect data to must-gather. 2026-09-19T01:15:53.231959337Z No resources found 2026-09-19T01:15:53.272898704Z error: the server doesn't have a resource type "performanceprofile" 2026-09-19T01:15:53.319048157Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-mxjcg 2026-09-19T01:15:53.319115047Z insights-runtime-extractor-9qdqw 2026-09-19T01:15:53.319136521Z insights-runtime-extractor-ktxj4 2026-09-19T01:15:53.319155662Z insights-runtime-extractor-nnwff 2026-09-19T01:15:53.417133751Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-19T01:15:53.480159323Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-19T01:15:53.482434683Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-19T01:15:53.661231226Z Wrote inspect data to must-gather. 2026-09-19T01:15:53.700687952Z ERROR: No running kube-apiserver pods found 2026-09-19T01:15:53.782122860Z INFO: "metallb-operator" not detected. Skipping. 2026-09-19T01:15:53.809790454Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-19T01:15:53.809790454Z See 'oc rsync -h' for help and examples 2026-09-19T01:15:53.832499667Z INFO: Worker host service log collection to complete. 2026-09-19T01:15:53.942213705Z INFO: 'previous-logs' folder not found on ip-10-0-132-189.ec2.internal, skipping... 2026-09-19T01:15:54.002511510Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-19T01:15:54.102885439Z Wrote inspect data to must-gather. 2026-09-19T01:15:54.325377617Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-19T01:15:54.535853343Z No resources found 2026-09-19T01:15:54.597274583Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-19T01:15:54.623992001Z Gathering data for ns/openshift-monitoring... 2026-09-19T01:15:54.643632441Z Wrote inspect data to must-gather. 2026-09-19T01:15:54.800581147Z Wrote inspect data to must-gather. 2026-09-19T01:15:54.859617209Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-19T01:15:54.864537319Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-19T01:15:54.923308919Z INFO: 'previous-logs' folder not found on ip-10-0-136-231.ec2.internal, skipping... 2026-09-19T01:15:54.960757220Z Wrote inspect data to must-gather. 2026-09-19T01:15:54.967104706Z error: the server doesn't have a resource type "machineconfigs" 2026-09-19T01:15:54.984676408Z INFO: Getting rules from prometheus-k8s-0 2026-09-19T01:15:55.021602905Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-19T01:15:55.031497846Z tar: Removing leading `/' from member names 2026-09-19T01:15:55.037208144Z INFO: HAProxy config collection complete. 2026-09-19T01:15:55.045786029Z Wrote inspect data to must-gather. 2026-09-19T01:15:55.134325956Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-19T01:15:55.197948252Z INFO: 'previous-logs' folder not found on ip-10-0-139-215.ec2.internal, skipping... 2026-09-19T01:15:55.198057559Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-19T01:15:55.198154043Z INFO: Machine Config Daemon termination log collection complete. 2026-09-19T01:15:55.292511005Z Wrote inspect data to must-gather. 2026-09-19T01:15:55.341922335Z Wrote inspect data to must-gather. 2026-09-19T01:15:55.377951015Z INFO: Getting status/config from prometheus-k8s-0 2026-09-19T01:15:55.497105124Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-19T01:15:55.537633525Z Wrote inspect data to must-gather. 2026-09-19T01:15:55.702011157Z Wrote inspect data to must-gather. 2026-09-19T01:15:55.751286848Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-19T01:15:55.774113336Z Wrote inspect data to must-gather. 2026-09-19T01:15:55.896446837Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-19T01:15:55.908349177Z Wrote inspect data to must-gather. 2026-09-19T01:15:55.954796891Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-19T01:15:56.057688999Z INFO: INTERCONNECT MODE 2026-09-19T01:15:56.057777591Z INFO: Gathering ovn-kubernetes DBs 2026-09-19T01:15:56.075686895Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:56.120477119Z Wrote inspect data to must-gather. 2026-09-19T01:15:56.190888603Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-19T01:15:56.266942033Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-19T01:15:56.293433908Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:56.296282264Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-19T01:15:56.304185050Z INFO: Gathering OVN_Northbound from ovnkube-node-9qx6x... 2026-09-19T01:15:56.304501686Z INFO: Gathering OVN_Northbound from ovnkube-node-srj9n... 2026-09-19T01:15:56.304733938Z INFO: Gathering OVN_Northbound from ovnkube-node-v7ckw... 2026-09-19T01:15:56.305043599Z INFO: Gathering OVN_Southbound from ovnkube-node-9qx6x... 2026-09-19T01:15:56.305335156Z INFO: Gathering OVN_Southbound from ovnkube-node-srj9n... 2026-09-19T01:15:56.305630629Z INFO: Gathering OVN_Southbound from ovnkube-node-v7ckw... 2026-09-19T01:15:56.341806047Z Wrote inspect data to must-gather. 2026-09-19T01:15:56.486076164Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-19T01:15:56.619941525Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:56.653508521Z tar: Removing leading `/' from member names 2026-09-19T01:15:56.695258232Z tar: Removing leading `/' from member names 2026-09-19T01:15:56.705565929Z tar: Removing leading `/' from member names 2026-09-19T01:15:56.736535562Z tar: Removing leading `/' from member names 2026-09-19T01:15:56.740975651Z tar: Removing leading `/' from member names 2026-09-19T01:15:56.748526233Z tar: Removing leading `/' from member names 2026-09-19T01:15:56.770626719Z Wrote inspect data to must-gather. 2026-09-19T01:15:56.848146915Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:56.851757867Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-19T01:15:57.016312287Z INFO: Getting status from alertmanager-main-0 2026-09-19T01:15:57.022890470Z W0919 01:15:57.022849 1482 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-19T01:15:57.030119034Z Wrote inspect data to must-gather. 2026-09-19T01:15:57.090885324Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:57.093300829Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-19T01:15:57.131331530Z INFO: Waiting for network log collection to complete ... 2026-09-19T01:15:57.131392611Z INFO: Waiting for ovnk database copies to complete ... 2026-09-19T01:15:57.131820361Z INFO: Copying ovnk databases complete. 2026-09-19T01:15:57.133613672Z 3.1M must-gather/network_logs/ovnk_database_store 2026-09-19T01:15:57.136329622Z ovnk_database_store/ 2026-09-19T01:15:57.136393428Z ovnk_database_store/ovnkube-node-v7ckw_nbdb 2026-09-19T01:15:57.150533806Z ovnk_database_store/ovnkube-node-srj9n_nbdb 2026-09-19T01:15:57.159778497Z ovnk_database_store/ovnkube-node-9qx6x_nbdb 2026-09-19T01:15:57.169707288Z ovnk_database_store/ovnkube-node-srj9n_sbdb 2026-09-19T01:15:57.192804701Z ovnk_database_store/ovnkube-node-v7ckw_sbdb 2026-09-19T01:15:57.212501575Z ovnk_database_store/ovnkube-node-9qx6x_sbdb 2026-09-19T01:15:57.236547583Z INFO: Network log collection complete. 2026-09-19T01:15:57.307171411Z Gathering data for ns/openshift-network-console... 2026-09-19T01:15:57.310859195Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:57.313239097Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-19T01:15:57.331315925Z Wrote inspect data to must-gather. 2026-09-19T01:15:57.528418638Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:57.581972534Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-19T01:15:57.583874909Z INFO: Fallback to identify the container image from release info 2026-09-19T01:15:57.724070752Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:57.726715619Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-19T01:15:57.764089348Z Gathering data for ns/openshift-console-operator... 2026-09-19T01:15:57.884073142Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:58.051076125Z Wrote inspect data to /must-gather/istio. 2026-09-19T01:15:58.141065010Z Gathering data for ns/openshift-console... 2026-09-19T01:15:58.265752538Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea 2026-09-19T01:15:58.376758443Z daemonset.apps/perf-node-gather-daemonset created 2026-09-19T01:15:58.510008354Z Waiting for performance profile collector pods to become ready: 1 2026-09-19T01:15:58.675398502Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-19T01:15:58.700132816Z Done executing Istio gather script 2026-09-19T01:15:59.202004433Z Gathering data for ns/openshift-dns-operator... 2026-09-19T01:15:59.453015560Z Gathering data for ns/openshift-dns... 2026-09-19T01:15:59.610454532Z Waiting for performance profile collector pods to become ready: 2 2026-09-19T01:15:59.875127591Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-19T01:16:00.029119161Z Gathering data for ns/openshift-image-registry... 2026-09-19T01:16:00.463438454Z Gathering data for ns/openshift-ingress-operator... 2026-09-19T01:16:00.727076351Z Waiting for performance profile collector pods to become ready: 3 2026-09-19T01:16:00.799946106Z Gathering data for ns/openshift-ingress... 2026-09-19T01:16:01.121198244Z Gathering data for ns/openshift-ingress-canary... 2026-09-19T01:16:01.487220985Z Gathering data for ns/openshift-insights... 2026-09-19T01:16:01.843260034Z Waiting for performance profile collector pods to become ready: 4 2026-09-19T01:16:02.948302125Z Waiting for performance profile collector pods to become ready: 5 2026-09-19T01:16:03.292423595Z Gathering data for ns/openshift-jobset-operator... 2026-09-19T01:16:04.050581812Z Waiting for performance profile collector pods to become ready: 6 2026-09-19T01:16:04.068272982Z Gathering data for ns/openshift-config... 2026-09-19T01:16:04.321873607Z Gathering data for ns/openshift-config-managed... 2026-09-19T01:16:04.834923578Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-19T01:16:04.883228942Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-19T01:16:05.089582369Z Gathering data for ns/openshift-kube-apiserver... 2026-09-19T01:16:05.169543835Z Waiting for performance profile collector pods to become ready: 7 2026-09-19T01:16:05.340023308Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-19T01:16:05.647352632Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-19T01:16:05.905728159Z Gathering data for ns/openshift-kube-scheduler... 2026-09-19T01:16:06.189398458Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-19T01:16:06.288460062Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-19T01:16:06.512609208Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-19T01:16:06.535809411Z Collecting performance related data for node ip-10-0-136-231.ec2.internal 2026-09-19T01:16:06.542144292Z Collecting performance related data for node ip-10-0-132-189.ec2.internal 2026-09-19T01:16:06.549378625Z Collecting performance related data for node ip-10-0-139-215.ec2.internal 2026-09-19T01:16:06.975331626Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-19T01:16:07.344427794Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-19T01:16:07.946916314Z Gathering data for ns/openshift-multus... 2026-09-19T01:16:09.408741582Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-19T01:16:09.887626684Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-19T01:16:10.570660210Z Gathering data for ns/openshift-host-network... 2026-09-19T01:16:10.879859350Z Gathering data for ns/openshift-network-diagnostics... 2026-09-19T01:16:11.402300300Z Gathering data for ns/openshift-network-node-identity... 2026-09-19T01:16:11.865767430Z Gathering data for ns/openshift-network-operator... 2026-09-19T01:16:12.351130213Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-19T01:16:12.640509529Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-19T01:16:13.099604255Z Gathering data for ns/openshift-apiserver-operator... 2026-09-19T01:16:13.403443911Z Gathering data for ns/openshift-apiserver... 2026-09-19T01:16:13.793238051Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-19T01:16:14.052953531Z Gathering data for ns/openshift-controller-manager... 2026-09-19T01:16:14.071427064Z Collecting kubelet logs for node ip-10-0-136-231.ec2.internal 2026-09-19T01:16:14.073177189Z Collecting kubelet logs for node ip-10-0-132-189.ec2.internal 2026-09-19T01:16:14.074694270Z Collecting kubelet logs for node ip-10-0-139-215.ec2.internal 2026-09-19T01:16:14.415477038Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-7kgft namespace 2026-09-19T01:16:14.419892278Z INFO: Node performance data collection complete. 2026-09-19T01:16:14.461817165Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-19T01:16:14.893947764Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-09-19T01:16:15.094665934Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-19T01:16:15.364237687Z Gathering data for ns/openshift-service-ca-operator... 2026-09-19T01:16:15.635335980Z Gathering data for ns/openshift-service-ca... 2026-09-19T01:16:15.949993459Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-19T01:16:16.776295677Z Wrote inspect data to must-gather. 2026-09-19T01:16:16.776336321Z error: inspection completed with the errors occurred while gathering data: 2026-09-19T01:16:16.776336321Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-19T01:16:16.890425899Z error: the server doesn't have a resource type "clusters" 2026-09-19T01:16:17.021319504Z Caches written to disk