2026-07-14T05:02:56.519996994Z [disk usage checker] Started 2026-07-14T05:02:56.523884331Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T05:02:56.716205299Z Gathering data for ns/openshift-cluster-version... 2026-07-14T05:02:56.828392193Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-14T05:02:56.902972406Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-14T05:02:56.972334256Z Gathering data for ns/default... 2026-07-14T05:02:57.221861925Z Gathering data for ns/openshift... 2026-07-14T05:02:57.602505982Z Gathering data for ns/kube-system... 2026-07-14T05:02:58.268903398Z Gathering data for ns/openshift-etcd... 2026-07-14T05:02:58.540783806Z Wrote inspect data to must-gather. 2026-07-14T05:02:58.540816565Z error: inspection completed with the errors occurred while gathering data: 2026-07-14T05:02:58.540816565Z namespaces "assisted-installer" not found 2026-07-14T05:02:59.054390793Z Waiting on subprocesses to finish execution. 2026-07-14T05:02:59.068988877Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-14T05:02:59.072074362Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-14T05:02:59.076570498Z INFO: Waiting for node performance related collection to complete ... 2026-07-14T05:02:59.077494608Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-14T05:02:59.078690897Z INFO: Collecting host service logs for crio 2026-07-14T05:02:59.078979425Z INFO: Collecting host service logs for kubelet 2026-07-14T05:02:59.079330129Z INFO: Collecting host service logs for rpm-ostreed 2026-07-14T05:02:59.079638680Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-14T05:02:59.080151197Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-14T05:02:59.080474019Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-14T05:02:59.080787278Z INFO: Collecting host service logs for NetworkManager 2026-07-14T05:02:59.081119475Z INFO: Collecting host service logs for openvswitch 2026-07-14T05:02:59.081409770Z INFO: Collecting host service logs for ovs-configuration 2026-07-14T05:02:59.081709266Z INFO: Collecting host service logs for ovsdb-server 2026-07-14T05:02:59.082017329Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-14T05:02:59.082388272Z INFO: Waiting for worker host service log collection to complete ... 2026-07-14T05:02:59.092302010Z INFO: Gathering HAProxy config files 2026-07-14T05:02:59.098951577Z Executing Istio gather script 2026-07-14T05:02:59.723547035Z INFO: "metallb-operator" not detected. Skipping. 2026-07-14T05:02:59.768080652Z No resources found in openshift-etcd namespace. 2026-07-14T05:03:00.137078356Z Wrote inspect data to must-gather. 2026-07-14T05:03:00.232153617Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-14T05:03:00.274931377Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-14T05:03:00.298216915Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-pl9hz 2026-07-14T05:03:00.298273738Z insights-runtime-extractor-ccs89 2026-07-14T05:03:00.298293505Z insights-runtime-extractor-lml9l 2026-07-14T05:03:00.298315277Z insights-runtime-extractor-sv5sn 2026-07-14T05:03:00.404636208Z No resources found 2026-07-14T05:03:00.416287828Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-14T05:03:00.416405996Z INFO: on-disk MachineConfig config collection complete. 2026-07-14T05:03:00.608711357Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-14T05:03:00.725458544Z No resources found 2026-07-14T05:03:00.739470327Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-14T05:03:00.968641370Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-14T05:03:00.968641370Z See 'oc rsync -h' for help and examples 2026-07-14T05:03:01.099807315Z error: the server doesn't have a resource type "performanceprofile" 2026-07-14T05:03:01.117227769Z ERROR: No running kube-apiserver pods found 2026-07-14T05:03:01.122982733Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.132583563Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-14T05:03:01.208528757Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.234888784Z INFO: Worker host service log collection to complete. 2026-07-14T05:03:01.292969590Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.315854719Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-14T05:03:01.348426818Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-14T05:03:01.369249099Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.399366815Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-14T05:03:01.438396958Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.541997280Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T05:03:01.581473963Z INFO: 'previous-logs' folder not found on ip-10-0-130-136.ec2.internal, skipping... 2026-07-14T05:03:01.605066423Z Gathering data for ns/openshift-monitoring... 2026-07-14T05:03:01.645368834Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.646932350Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.688966123Z tar: Removing leading `/' from member names 2026-07-14T05:03:01.694531019Z INFO: HAProxy config collection complete. 2026-07-14T05:03:01.839572378Z error: the server doesn't have a resource type "machineconfigs" 2026-07-14T05:03:01.902829909Z INFO: 'previous-logs' folder not found on ip-10-0-130-137.ec2.internal, skipping... 2026-07-14T05:03:01.944520954Z Wrote inspect data to must-gather. 2026-07-14T05:03:01.951853315Z INFO: Getting rules from prometheus-k8s-0 2026-07-14T05:03:01.979634860Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-14T05:03:02.034713909Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-14T05:03:02.197616398Z INFO: 'previous-logs' folder not found on ip-10-0-140-216.ec2.internal, skipping... 2026-07-14T05:03:02.197685407Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-14T05:03:02.197718068Z INFO: Machine Config Daemon termination log collection complete. 2026-07-14T05:03:02.213149632Z Wrote inspect data to must-gather. 2026-07-14T05:03:02.286328480Z Wrote inspect data to must-gather. 2026-07-14T05:03:02.376191920Z INFO: Getting status/config from prometheus-k8s-0 2026-07-14T05:03:02.447751568Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-14T05:03:02.515938088Z INFO: INTERCONNECT MODE 2026-07-14T05:03:02.515972298Z INFO: Gathering ovn-kubernetes DBs 2026-07-14T05:03:02.628232473Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-14T05:03:02.653233177Z Wrote inspect data to must-gather. 2026-07-14T05:03:02.675257494Z INFO: Gathering OVN_Northbound from ovnkube-node-4brrv... 2026-07-14T05:03:02.675465260Z INFO: Gathering OVN_Northbound from ovnkube-node-9lwrk... 2026-07-14T05:03:02.675754047Z INFO: Gathering OVN_Northbound from ovnkube-node-vhf6d... 2026-07-14T05:03:02.676336193Z INFO: Gathering OVN_Southbound from ovnkube-node-4brrv... 2026-07-14T05:03:02.676739286Z INFO: Gathering OVN_Southbound from ovnkube-node-9lwrk... 2026-07-14T05:03:02.677048692Z INFO: Gathering OVN_Southbound from ovnkube-node-vhf6d... 2026-07-14T05:03:02.942406831Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:02.985441469Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-14T05:03:03.002450934Z tar: Removing leading `/' from member names 2026-07-14T05:03:03.003059973Z tar: Removing leading `/' from member names 2026-07-14T05:03:03.012731418Z tar: Removing leading `/' from member names 2026-07-14T05:03:03.029341331Z tar: Removing leading `/' from member names 2026-07-14T05:03:03.048372672Z tar: Removing leading `/' from member names 2026-07-14T05:03:03.065047265Z Wrote inspect data to must-gather. 2026-07-14T05:03:03.108171311Z tar: Removing leading `/' from member names 2026-07-14T05:03:03.175041958Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-14T05:03:03.226688531Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:03.230225012Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-14T05:03:03.277322840Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-14T05:03:03.310852920Z Wrote inspect data to must-gather. 2026-07-14T05:03:03.475491769Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:03.520692718Z INFO: Waiting for network log collection to complete ... 2026-07-14T05:03:03.520818235Z INFO: Waiting for ovnk database copies to complete ... 2026-07-14T05:03:03.521346935Z INFO: Copying ovnk databases complete. 2026-07-14T05:03:03.523609630Z 2.6M must-gather/network_logs/ovnk_database_store 2026-07-14T05:03:03.525994592Z ovnk_database_store/ 2026-07-14T05:03:03.525994592Z ovnk_database_store/ovnkube-node-vhf6d_nbdb 2026-07-14T05:03:03.533932076Z ovnk_database_store/ovnkube-node-4brrv_nbdb 2026-07-14T05:03:03.540018353Z ovnk_database_store/ovnkube-node-vhf6d_sbdb 2026-07-14T05:03:03.551817893Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-14T05:03:03.556568937Z Wrote inspect data to must-gather. 2026-07-14T05:03:03.557879267Z ovnk_database_store/ovnkube-node-4brrv_sbdb 2026-07-14T05:03:03.575067179Z ovnk_database_store/ovnkube-node-9lwrk_sbdb 2026-07-14T05:03:03.592568983Z ovnk_database_store/ovnkube-node-9lwrk_nbdb 2026-07-14T05:03:03.601143262Z INFO: Network log collection complete. 2026-07-14T05:03:03.705854714Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:03.709032627Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-14T05:03:03.759088212Z W0714 05:03:03.759039 1465 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-14T05:03:03.764326092Z Wrote inspect data to must-gather. 2026-07-14T05:03:03.807625708Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-14T05:03:03.835017962Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-14T05:03:03.900611660Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:04.036876247Z Gathering data for ns/openshift-network-console... 2026-07-14T05:03:04.091921609Z Wrote inspect data to must-gather. 2026-07-14T05:03:04.126367484Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:04.129276835Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-14T05:03:04.243553720Z INFO: Getting status from alertmanager-main-0 2026-07-14T05:03:04.386054495Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:04.439291040Z Wrote inspect data to must-gather. 2026-07-14T05:03:04.477929826Z Gathering data for ns/openshift-console-operator... 2026-07-14T05:03:04.598211318Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:04.601349604Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-14T05:03:04.704911573Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-14T05:03:04.706990838Z INFO: Fallback to identify the container image from release info 2026-07-14T05:03:04.763279096Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:04.871012417Z Gathering data for ns/openshift-console... 2026-07-14T05:03:04.950462716Z Wrote inspect data to /must-gather/istio. 2026-07-14T05:03:05.367226363Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-14T05:03:05.435889724Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d 2026-07-14T05:03:05.563748113Z daemonset.apps/perf-node-gather-daemonset created 2026-07-14T05:03:05.625255168Z Done executing Istio gather script 2026-07-14T05:03:05.697727627Z Waiting for performance profile collector pods to become ready: 1 2026-07-14T05:03:05.836552854Z Gathering data for ns/openshift-dns-operator... 2026-07-14T05:03:06.131805400Z Gathering data for ns/openshift-dns... 2026-07-14T05:03:06.549386456Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T05:03:06.638914667Z Gathering data for ns/openshift-image-registry... 2026-07-14T05:03:06.813785885Z Waiting for performance profile collector pods to become ready: 2 2026-07-14T05:03:07.056983915Z Gathering data for ns/openshift-ingress-operator... 2026-07-14T05:03:07.379214533Z Gathering data for ns/openshift-ingress... 2026-07-14T05:03:07.657867303Z Gathering data for ns/openshift-ingress-canary... 2026-07-14T05:03:07.929207403Z Waiting for performance profile collector pods to become ready: 3 2026-07-14T05:03:07.995429163Z Gathering data for ns/openshift-insights... 2026-07-14T05:03:09.039542714Z Waiting for performance profile collector pods to become ready: 4 2026-07-14T05:03:09.698719824Z Gathering data for ns/openshift-config... 2026-07-14T05:03:09.991088287Z Gathering data for ns/openshift-config-managed... 2026-07-14T05:03:10.149911843Z Waiting for performance profile collector pods to become ready: 5 2026-07-14T05:03:10.351886207Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-14T05:03:10.570027018Z Gathering data for ns/openshift-kube-apiserver... 2026-07-14T05:03:10.788089282Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-14T05:03:11.021427722Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-14T05:03:11.280212938Z Waiting for performance profile collector pods to become ready: 6 2026-07-14T05:03:11.292138870Z Gathering data for ns/openshift-kube-scheduler... 2026-07-14T05:03:11.496514473Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-14T05:03:11.557009755Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T05:03:11.746471995Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-14T05:03:12.017917046Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-14T05:03:12.286443137Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-14T05:03:12.403295769Z Waiting for performance profile collector pods to become ready: 7 2026-07-14T05:03:12.803903706Z Gathering data for ns/openshift-multus... 2026-07-14T05:03:13.516319401Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-14T05:03:13.750886769Z Collecting performance related data for node ip-10-0-130-136.ec2.internal 2026-07-14T05:03:13.759008478Z Collecting performance related data for node ip-10-0-140-216.ec2.internal 2026-07-14T05:03:13.769854726Z Collecting performance related data for node ip-10-0-130-137.ec2.internal 2026-07-14T05:03:14.146384948Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-14T05:03:15.341112300Z Gathering data for ns/openshift-host-network... 2026-07-14T05:03:15.602554671Z Gathering data for ns/openshift-network-diagnostics... 2026-07-14T05:03:16.086943836Z Gathering data for ns/openshift-network-node-identity... 2026-07-14T05:03:16.478210864Z Gathering data for ns/openshift-network-operator... 2026-07-14T05:03:16.564047783Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T05:03:16.840181555Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-14T05:03:17.054520558Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-14T05:03:17.475540281Z Gathering data for ns/openshift-apiserver-operator... 2026-07-14T05:03:17.748321968Z Gathering data for ns/openshift-apiserver... 2026-07-14T05:03:18.081437003Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-14T05:03:18.324785700Z Gathering data for ns/openshift-controller-manager... 2026-07-14T05:03:18.566306636Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-14T05:03:19.179204164Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-14T05:03:19.430086310Z Gathering data for ns/openshift-service-ca-operator... 2026-07-14T05:03:19.700685853Z Gathering data for ns/openshift-service-ca... 2026-07-14T05:03:19.989930163Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-14T05:03:20.827536771Z Wrote inspect data to must-gather. 2026-07-14T05:03:20.827589139Z error: inspection completed with the errors occurred while gathering data: 2026-07-14T05:03:20.827589139Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-14T05:03:21.569241294Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-14T05:03:21.692193199Z Collecting kubelet logs for node ip-10-0-130-136.ec2.internal 2026-07-14T05:03:21.694067236Z Collecting kubelet logs for node ip-10-0-140-216.ec2.internal 2026-07-14T05:03:21.695900671Z Collecting kubelet logs for node ip-10-0-130-137.ec2.internal 2026-07-14T05:03:21.985647445Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-p7kh9 namespace 2026-07-14T05:03:21.990128269Z INFO: Node performance data collection complete. 2026-07-14T05:03:22.111350418Z error: the server doesn't have a resource type "clusters" 2026-07-14T05:03:22.272429616Z Caches written to disk