2026-07-01T19:39:01.951011070Z [disk usage checker] Started
2026-07-01T19:39:01.954897296Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-01T19:39:02.166164384Z Gathering data for ns/openshift-cluster-version...
2026-07-01T19:39:02.318694919Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-01T19:39:02.419410641Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-01T19:39:02.512183353Z Gathering data for ns/default...
2026-07-01T19:39:02.843006693Z Gathering data for ns/openshift...
2026-07-01T19:39:03.302766120Z Gathering data for ns/kube-system...
2026-07-01T19:39:04.036780894Z Gathering data for ns/openshift-etcd...
2026-07-01T19:39:04.357204056Z Wrote inspect data to must-gather.
2026-07-01T19:39:04.357240509Z error: inspection completed with the errors occurred while gathering data:
2026-07-01T19:39:04.357240509Z     namespaces "assisted-installer" not found
2026-07-01T19:39:04.653627528Z Waiting on subprocesses to finish execution.
2026-07-01T19:39:04.658643369Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-01T19:39:04.668950530Z INFO: Collecting host service logs for crio
2026-07-01T19:39:04.669222600Z INFO: Collecting host service logs for kubelet
2026-07-01T19:39:04.669566625Z INFO: Collecting host service logs for rpm-ostreed
2026-07-01T19:39:04.669864703Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-01T19:39:04.670431565Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-01T19:39:04.670952128Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-01T19:39:04.671310470Z INFO: Collecting host service logs for NetworkManager
2026-07-01T19:39:04.671666827Z INFO: Collecting host service logs for openvswitch
2026-07-01T19:39:04.672033750Z INFO: Collecting host service logs for ovs-configuration
2026-07-01T19:39:04.672374763Z INFO: Collecting host service logs for ovsdb-server
2026-07-01T19:39:04.672745155Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-01T19:39:04.673263326Z INFO: Waiting for worker host service log collection to complete ...
2026-07-01T19:39:04.676691000Z INFO: Gathering HAProxy config files
2026-07-01T19:39:04.698608722Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-01T19:39:04.703696146Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-01T19:39:04.718055396Z Executing Istio gather script
2026-07-01T19:39:04.853635022Z INFO: Waiting for node performance related collection to complete ...
2026-07-01T19:39:05.334177246Z INFO: Collecting Insights Archives from insights-operator-5f67b55555-l2ksf
2026-07-01T19:39:05.334225750Z insights-runtime-extractor-946h2
2026-07-01T19:39:05.334246023Z insights-runtime-extractor-9hwsp
2026-07-01T19:39:05.334265651Z insights-runtime-extractor-vtzfj
2026-07-01T19:39:05.455214727Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-01T19:39:05.582040975Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-01T19:39:05.582131336Z INFO: on-disk MachineConfig config collection complete.
2026-07-01T19:39:05.825887865Z Wrote inspect data to must-gather.
2026-07-01T19:39:05.924167014Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-01T19:39:05.963158103Z No resources found
2026-07-01T19:39:05.996983313Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-01T19:39:06.113216716Z No resources found in openshift-etcd namespace.
2026-07-01T19:39:06.120216060Z INFO: "metallb-operator" not detected. Skipping.
2026-07-01T19:39:06.261159272Z error: the server doesn't have a resource type "performanceprofile"
2026-07-01T19:39:06.268727282Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-01T19:39:06.504496720Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-01T19:39:06.504496720Z See 'oc rsync -h' for help and examples
2026-07-01T19:39:06.606066731Z Wrote inspect data to must-gather.
2026-07-01T19:39:06.606703165Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-01T19:39:06.671062070Z Wrote inspect data to must-gather.
2026-07-01T19:39:06.749650506Z No resources found
2026-07-01T19:39:06.787455218Z Wrote inspect data to must-gather.
2026-07-01T19:39:06.840631551Z ERROR: No running kube-apiserver pods found
2026-07-01T19:39:06.864334722Z INFO: Worker host service log collection to complete.
2026-07-01T19:39:06.942705357Z Wrote inspect data to must-gather.
2026-07-01T19:39:06.948495391Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-01T19:39:06.963355218Z Wrote inspect data to must-gather.
2026-07-01T19:39:06.966344442Z Wrote inspect data to must-gather.
2026-07-01T19:39:07.003634069Z INFO: 'previous-logs' folder not found on ip-10-0-131-188.ec2.internal, skipping...
2026-07-01T19:39:07.009929571Z tar: Removing leading `/' from member names
2026-07-01T19:39:07.012786800Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-01T19:39:07.022884315Z INFO: HAProxy config collection complete.
2026-07-01T19:39:07.041908822Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-01T19:39:07.055865463Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-01T19:39:07.101672845Z Gathering data for ns/openshift-monitoring...
2026-07-01T19:39:07.230400622Z error: the server doesn't have a resource type "machineconfigs"
2026-07-01T19:39:07.342753854Z Wrote inspect data to must-gather.
2026-07-01T19:39:07.382630136Z INFO: 'previous-logs' folder not found on ip-10-0-134-10.ec2.internal, skipping...
2026-07-01T19:39:07.414878039Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-01T19:39:07.580124053Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-01T19:39:07.604286392Z INFO: Getting rules from prometheus-k8s-0
2026-07-01T19:39:07.647517443Z Wrote inspect data to must-gather.
2026-07-01T19:39:07.688724767Z Wrote inspect data to must-gather.
2026-07-01T19:39:07.694417883Z INFO: 'previous-logs' folder not found on ip-10-0-142-220.ec2.internal, skipping...
2026-07-01T19:39:07.694448991Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-01T19:39:07.694536810Z INFO: Machine Config Daemon termination log collection complete.
2026-07-01T19:39:07.848337628Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-01T19:39:07.917743209Z Wrote inspect data to must-gather.
2026-07-01T19:39:08.003767316Z INFO: Getting status/config from prometheus-k8s-0
2026-07-01T19:39:08.098063430Z Wrote inspect data to must-gather.
2026-07-01T19:39:08.209294561Z INFO: INTERCONNECT MODE
2026-07-01T19:39:08.209339728Z INFO: Gathering ovn-kubernetes DBs
2026-07-01T19:39:08.292165496Z Wrote inspect data to must-gather.
2026-07-01T19:39:08.345354525Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-01T19:39:08.390119088Z INFO: Gathering OVN_Northbound from ovnkube-node-b9kmc...
2026-07-01T19:39:08.390354007Z INFO: Gathering OVN_Northbound from ovnkube-node-dgm54...
2026-07-01T19:39:08.390618313Z INFO: Gathering OVN_Northbound from ovnkube-node-jtrfg...
2026-07-01T19:39:08.390952994Z INFO: Gathering OVN_Southbound from ovnkube-node-b9kmc...
2026-07-01T19:39:08.391215760Z INFO: Gathering OVN_Southbound from ovnkube-node-dgm54...
2026-07-01T19:39:08.391528269Z INFO: Gathering OVN_Southbound from ovnkube-node-jtrfg...
2026-07-01T19:39:08.431662966Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-01T19:39:08.652664710Z Wrote inspect data to must-gather.
2026-07-01T19:39:08.679289076Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:08.712876761Z tar: Removing leading `/' from member names
2026-07-01T19:39:08.742242596Z tar: Removing leading `/' from member names
2026-07-01T19:39:08.743732704Z tar: Removing leading `/' from member names
2026-07-01T19:39:08.752237806Z tar: Removing leading `/' from member names
2026-07-01T19:39:08.774940988Z tar: Removing leading `/' from member names
2026-07-01T19:39:08.781398307Z tar: Removing leading `/' from member names
2026-07-01T19:39:08.812000038Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-01T19:39:08.891754370Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-01T19:39:08.933219977Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:08.936299538Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-01T19:39:08.941419406Z Wrote inspect data to must-gather.
2026-07-01T19:39:09.075937744Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-01T19:39:09.179124737Z Wrote inspect data to must-gather.
2026-07-01T19:39:09.180657717Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:09.305820237Z INFO: Waiting for network log collection to complete ...
2026-07-01T19:39:09.305864153Z INFO: Waiting for ovnk database copies to complete ...
2026-07-01T19:39:09.306373585Z INFO: Copying ovnk databases complete.
2026-07-01T19:39:09.309699302Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-01T19:39:09.313085760Z ovnk_database_store/
2026-07-01T19:39:09.313141912Z ovnk_database_store/ovnkube-node-b9kmc_sbdb
2026-07-01T19:39:09.331711178Z ovnk_database_store/ovnkube-node-jtrfg_sbdb
2026-07-01T19:39:09.346388134Z ovnk_database_store/ovnkube-node-b9kmc_nbdb
2026-07-01T19:39:09.353892302Z ovnk_database_store/ovnkube-node-jtrfg_nbdb
2026-07-01T19:39:09.360430772Z ovnk_database_store/ovnkube-node-dgm54_nbdb
2026-07-01T19:39:09.368881865Z ovnk_database_store/ovnkube-node-dgm54_sbdb
2026-07-01T19:39:09.387711423Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-01T19:39:09.389036953Z INFO: Network log collection complete.
2026-07-01T19:39:09.417182646Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:09.420427277Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-01T19:39:09.426154855Z W0701 19:39:09.426105    1470 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-01T19:39:09.431642394Z Wrote inspect data to must-gather.
2026-07-01T19:39:09.556837999Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-01T19:39:09.619597167Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:09.726085122Z Wrote inspect data to must-gather.
2026-07-01T19:39:09.753086028Z Gathering data for ns/openshift-network-console...
2026-07-01T19:39:09.790107062Z INFO: Getting status from alertmanager-main-0
2026-07-01T19:39:09.832089703Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:09.835657448Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-01T19:39:10.020411137Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-01T19:39:10.022729449Z INFO: Fallback to identify the container image from release info
2026-07-01T19:39:10.086460510Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:10.223216791Z Gathering data for ns/openshift-console-operator...
2026-07-01T19:39:10.290441767Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:10.293207616Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-01T19:39:10.455811779Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:10.632397766Z Gathering data for ns/openshift-console...
2026-07-01T19:39:10.634050249Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2c4f9949a07a5338dd10a5abcb8d6d3f5eebe93333139869f9eec8802800162f
2026-07-01T19:39:10.657456810Z Wrote inspect data to /must-gather/istio.
2026-07-01T19:39:10.769257008Z daemonset.apps/perf-node-gather-daemonset created
2026-07-01T19:39:10.901289719Z Waiting for performance profile collector pods to become ready: 1
2026-07-01T19:39:11.214874128Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-01T19:39:11.365673263Z Done executing Istio gather script
2026-07-01T19:39:11.721249704Z Gathering data for ns/openshift-dns-operator...
2026-07-01T19:39:12.027213332Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-01T19:39:12.033504943Z Gathering data for ns/openshift-dns...
2026-07-01T19:39:12.036642817Z Waiting for performance profile collector pods to become ready: 2
2026-07-01T19:39:12.717906993Z Gathering data for ns/openshift-image-registry...
2026-07-01T19:39:13.158115182Z Waiting for performance profile collector pods to become ready: 3
2026-07-01T19:39:13.207265302Z Gathering data for ns/openshift-ingress-operator...
2026-07-01T19:39:13.544959915Z Gathering data for ns/openshift-ingress...
2026-07-01T19:39:13.866830486Z Gathering data for ns/openshift-ingress-canary...
2026-07-01T19:39:14.265862065Z Gathering data for ns/openshift-insights...
2026-07-01T19:39:14.283382613Z Waiting for performance profile collector pods to become ready: 4
2026-07-01T19:39:15.394871704Z Waiting for performance profile collector pods to become ready: 5
2026-07-01T19:39:16.134372378Z Gathering data for ns/openshift-config...
2026-07-01T19:39:16.397397156Z Gathering data for ns/openshift-config-managed...
2026-07-01T19:39:16.520002325Z Waiting for performance profile collector pods to become ready: 6
2026-07-01T19:39:16.847700278Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-01T19:39:17.033760983Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-01T19:39:17.118293736Z Gathering data for ns/openshift-kube-apiserver...
2026-07-01T19:39:17.382853880Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-01T19:39:17.642449768Z Waiting for performance profile collector pods to become ready: 7
2026-07-01T19:39:17.682988971Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-01T19:39:17.983318895Z Gathering data for ns/openshift-kube-scheduler...
2026-07-01T19:39:18.248019938Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-01T19:39:18.532772701Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-01T19:39:18.751882496Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-01T19:39:18.872886225Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-01T19:39:19.014881818Z Collecting performance related data for node ip-10-0-134-10.ec2.internal
2026-07-01T19:39:19.022829634Z Collecting performance related data for node ip-10-0-142-220.ec2.internal
2026-07-01T19:39:19.030646803Z Collecting performance related data for node ip-10-0-131-188.ec2.internal
2026-07-01T19:39:19.223141016Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-01T19:39:19.810883543Z Gathering data for ns/openshift-multus...
2026-07-01T19:39:21.363908324Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-01T19:39:22.039500284Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-01T19:39:22.740102000Z Gathering data for ns/openshift-host-network...
2026-07-01T19:39:23.055062935Z Gathering data for ns/openshift-network-diagnostics...
2026-07-01T19:39:23.632632004Z Gathering data for ns/openshift-network-node-identity...
2026-07-01T19:39:24.053825144Z Gathering data for ns/openshift-network-operator...
2026-07-01T19:39:24.520441899Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-01T19:39:24.817763149Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-01T19:39:25.360416762Z Gathering data for ns/openshift-apiserver-operator...
2026-07-01T19:39:25.719048295Z Gathering data for ns/openshift-apiserver...
2026-07-01T19:39:26.138760230Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-01T19:39:26.485017681Z Gathering data for ns/openshift-controller-manager...
2026-07-01T19:39:26.750013096Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-01T19:39:27.045371947Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-01T19:39:27.289876872Z Collecting kubelet logs for node ip-10-0-134-10.ec2.internal
2026-07-01T19:39:27.292097833Z Collecting kubelet logs for node ip-10-0-142-220.ec2.internal
2026-07-01T19:39:27.295276671Z Collecting kubelet logs for node ip-10-0-131-188.ec2.internal
2026-07-01T19:39:27.498833257Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-01T19:39:27.641983283Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-9xwfk namespace
2026-07-01T19:39:27.653170686Z INFO: Node performance data collection complete.
2026-07-01T19:39:27.894390158Z Gathering data for ns/openshift-service-ca-operator...
2026-07-01T19:39:28.219101190Z Gathering data for ns/openshift-service-ca...
2026-07-01T19:39:28.598131657Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-01T19:39:29.496296303Z Wrote inspect data to must-gather.
2026-07-01T19:39:29.496344930Z error: inspection completed with the errors occurred while gathering data:
2026-07-01T19:39:29.496344930Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-01T19:39:29.623088610Z error: the server doesn't have a resource type "clusters"
2026-07-01T19:39:29.753370620Z Caches written to disk