2026-09-16T19:48:32.335200024Z [disk usage checker] Started
2026-09-16T19:48:32.338439443Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:48:32.541153635Z Gathering data for ns/openshift-cluster-version...
2026-09-16T19:48:32.673574749Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:48:32.769203130Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:48:32.850185479Z Gathering data for ns/default...
2026-09-16T19:48:33.148556734Z Gathering data for ns/openshift...
2026-09-16T19:48:33.550072838Z Gathering data for ns/kube-system...
2026-09-16T19:48:34.209374412Z Gathering data for ns/openshift-etcd...
2026-09-16T19:48:34.542583973Z Wrote inspect data to must-gather.
2026-09-16T19:48:34.542616868Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:48:34.542616868Z     namespaces "assisted-installer" not found
2026-09-16T19:48:35.003549850Z Waiting on subprocesses to finish execution.
2026-09-16T19:48:35.011186923Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T19:48:35.012732788Z INFO: Gathering HAProxy config files
2026-09-16T19:48:35.013867978Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T19:48:35.015596616Z INFO: Collecting host service logs for crio
2026-09-16T19:48:35.015897969Z INFO: Collecting host service logs for kubelet
2026-09-16T19:48:35.016201384Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T19:48:35.016493581Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T19:48:35.016806860Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T19:48:35.017120377Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T19:48:35.017414252Z INFO: Collecting host service logs for NetworkManager
2026-09-16T19:48:35.017789676Z INFO: Collecting host service logs for openvswitch
2026-09-16T19:48:35.018082816Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T19:48:35.018399776Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T19:48:35.018720180Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T19:48:35.019081431Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T19:48:35.020893274Z Executing Istio gather script
2026-09-16T19:48:35.027948681Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T19:48:35.166168020Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T19:48:35.618336557Z No resources found
2026-09-16T19:48:35.700456499Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T19:48:35.741133762Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T19:48:35.769312504Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T19:48:35.946263642Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T19:48:35.946354177Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T19:48:36.169260514Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T19:48:36.242608252Z Wrote inspect data to must-gather.
2026-09-16T19:48:36.419032195Z INFO: Collecting Insights Archives from insights-operator-5844b856db-jczs9
2026-09-16T19:48:36.419032195Z insights-runtime-extractor-46w9f
2026-09-16T19:48:36.419032195Z insights-runtime-extractor-lggrm
2026-09-16T19:48:36.419032195Z insights-runtime-extractor-p96zq
2026-09-16T19:48:36.426053151Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T19:48:36.447720800Z No resources found in openshift-etcd namespace.
2026-09-16T19:48:36.460025888Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T19:48:36.479179809Z Wrote inspect data to must-gather.
2026-09-16T19:48:36.508330712Z INFO: Worker host service log collection to complete.
2026-09-16T19:48:36.617889683Z ERROR: No running kube-apiserver pods found
2026-09-16T19:48:36.627652823Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T19:48:36.627652823Z See 'oc rsync -h' for help and examples
2026-09-16T19:48:36.949477157Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T19:48:37.056839006Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:48:37.125137758Z No resources found
2026-09-16T19:48:37.164228662Z Wrote inspect data to must-gather.
2026-09-16T19:48:37.249042535Z Wrote inspect data to must-gather.
2026-09-16T19:48:37.260770327Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T19:48:37.280014032Z Gathering data for ns/openshift-monitoring...
2026-09-16T19:48:37.349827462Z Wrote inspect data to must-gather.
2026-09-16T19:48:37.427900271Z INFO: 'previous-logs' folder not found on ip-10-0-133-109.ec2.internal, skipping...
2026-09-16T19:48:37.434662122Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T19:48:37.439306252Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:48:37.442400040Z Wrote inspect data to must-gather.
2026-09-16T19:48:37.526710247Z Wrote inspect data to must-gather.
2026-09-16T19:48:37.600081026Z tar: Removing leading `/' from member names
2026-09-16T19:48:37.604640242Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T19:48:37.608183600Z INFO: HAProxy config collection complete.
2026-09-16T19:48:37.765157187Z Wrote inspect data to must-gather.
2026-09-16T19:48:37.797565250Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T19:48:37.821762076Z INFO: 'previous-logs' folder not found on ip-10-0-137-205.ec2.internal, skipping...
2026-09-16T19:48:38.010981617Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T19:48:38.084226442Z Wrote inspect data to must-gather.
2026-09-16T19:48:38.089719030Z Wrote inspect data to must-gather.
2026-09-16T19:48:38.118292096Z INFO: 'previous-logs' folder not found on ip-10-0-141-132.ec2.internal, skipping...
2026-09-16T19:48:38.118388832Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T19:48:38.118423426Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T19:48:38.261225168Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T19:48:38.364453982Z Wrote inspect data to must-gather.
2026-09-16T19:48:38.457845678Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T19:48:38.509524519Z Wrote inspect data to must-gather.
2026-09-16T19:48:38.716271463Z INFO: INTERCONNECT MODE
2026-09-16T19:48:38.716301264Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T19:48:38.755394122Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T19:48:38.766120126Z Wrote inspect data to must-gather.
2026-09-16T19:48:38.901395748Z INFO: Gathering OVN_Northbound from ovnkube-node-4dk5x...
2026-09-16T19:48:38.901775827Z INFO: Gathering OVN_Northbound from ovnkube-node-n96x4...
2026-09-16T19:48:38.902083120Z INFO: Gathering OVN_Northbound from ovnkube-node-zzhc8...
2026-09-16T19:48:38.902392337Z INFO: Gathering OVN_Southbound from ovnkube-node-4dk5x...
2026-09-16T19:48:38.902660284Z INFO: Gathering OVN_Southbound from ovnkube-node-n96x4...
2026-09-16T19:48:38.902963203Z INFO: Gathering OVN_Southbound from ovnkube-node-zzhc8...
2026-09-16T19:48:39.060963486Z Wrote inspect data to must-gather.
2026-09-16T19:48:39.072338161Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T19:48:39.148908645Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:48:39.258809067Z tar: Removing leading `/' from member names
2026-09-16T19:48:39.261315555Z tar: Removing leading `/' from member names
2026-09-16T19:48:39.276332096Z tar: Removing leading `/' from member names
2026-09-16T19:48:39.278576823Z tar: Removing leading `/' from member names
2026-09-16T19:48:39.312970158Z tar: Removing leading `/' from member names
2026-09-16T19:48:39.320487767Z tar: Removing leading `/' from member names
2026-09-16T19:48:39.398441727Z Wrote inspect data to must-gather.
2026-09-16T19:48:39.447251231Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T19:48:39.645580903Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T19:48:39.671593698Z Wrote inspect data to must-gather.
2026-09-16T19:48:39.724956488Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T19:48:39.817948893Z INFO: Waiting for network log collection to complete ...
2026-09-16T19:48:39.817983780Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T19:48:39.818431527Z INFO: Copying ovnk databases complete.
2026-09-16T19:48:39.820237197Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-16T19:48:39.823550439Z ovnk_database_store/
2026-09-16T19:48:39.823550439Z ovnk_database_store/ovnkube-node-4dk5x_nbdb
2026-09-16T19:48:39.832912319Z ovnk_database_store/ovnkube-node-zzhc8_nbdb
2026-09-16T19:48:39.843107163Z ovnk_database_store/ovnkube-node-zzhc8_sbdb
2026-09-16T19:48:39.861093541Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:39.861123389Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:48:39.863376166Z ovnk_database_store/ovnkube-node-n96x4_sbdb
2026-09-16T19:48:39.884189814Z ovnk_database_store/ovnkube-node-n96x4_nbdb
2026-09-16T19:48:39.890960912Z ovnk_database_store/ovnkube-node-4dk5x_sbdb
2026-09-16T19:48:39.912904056Z INFO: Network log collection complete.
2026-09-16T19:48:40.005085560Z Wrote inspect data to must-gather.
2026-09-16T19:48:40.005838576Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T19:48:40.045428246Z Gathering data for ns/openshift-network-console...
2026-09-16T19:48:40.087138520Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:40.090264762Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T19:48:40.303489440Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:40.316308338Z Wrote inspect data to must-gather.
2026-09-16T19:48:40.445707457Z INFO: Getting status from alertmanager-main-0
2026-09-16T19:48:40.547138112Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:40.550632126Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T19:48:40.568547027Z Gathering data for ns/openshift-console-operator...
2026-09-16T19:48:40.632968414Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T19:48:40.635659440Z INFO: Fallback to identify the container image from release info
2026-09-16T19:48:40.760898649Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:40.966472423Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:40.968932880Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T19:48:40.973324181Z Gathering data for ns/openshift-console...
2026-09-16T19:48:41.184789955Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T19:48:41.193364041Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:41.320945180Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T19:48:41.416276184Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:41.420882783Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-16T19:48:41.469296715Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T19:48:41.572873642Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T19:48:41.610400736Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:41.821335458Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:41.823619265Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-16T19:48:42.009934456Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:42.164101901Z Gathering data for ns/openshift-dns-operator...
2026-09-16T19:48:42.253544124Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:42.255912181Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T19:48:42.419505328Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:42.444925093Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:48:42.488194654Z Gathering data for ns/openshift-dns...
2026-09-16T19:48:42.604048733Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T19:48:42.606629518Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:48:43.142416235Z Gathering data for ns/openshift-image-registry...
2026-09-16T19:48:43.257822118Z Done executing Istio gather script
2026-09-16T19:48:43.618579525Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T19:48:43.735329522Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T19:48:43.954262891Z Gathering data for ns/openshift-ingress...
2026-09-16T19:48:44.294106572Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T19:48:44.699194654Z Gathering data for ns/openshift-insights...
2026-09-16T19:48:44.842793493Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T19:48:45.955195893Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T19:48:47.077883797Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T19:48:47.123274156Z Gathering data for ns/kserve...
2026-09-16T19:48:47.451396852Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:48:48.200518863Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T19:48:48.479804447Z Gathering data for ns/openshift-config...
2026-09-16T19:48:48.731218022Z Gathering data for ns/openshift-config-managed...
2026-09-16T19:48:49.213280020Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T19:48:49.327468441Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T19:48:49.545326384Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T19:48:49.610926804Z Collecting performance related data for node ip-10-0-137-205.ec2.internal
2026-09-16T19:48:49.620011007Z Collecting performance related data for node ip-10-0-141-132.ec2.internal
2026-09-16T19:48:49.627349256Z Collecting performance related data for node ip-10-0-133-109.ec2.internal
2026-09-16T19:48:49.854110216Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T19:48:50.265893021Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T19:48:50.660451803Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T19:48:51.007651184Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T19:48:51.344960788Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T19:48:51.677362443Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T19:48:52.034410740Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T19:48:52.457575503Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:48:52.786459326Z Gathering data for ns/openshift-multus...
2026-09-16T19:48:54.466076821Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T19:48:55.745937980Z Gathering data for ns/openshift-host-network...
2026-09-16T19:48:56.090419309Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T19:48:56.617362950Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T19:48:57.048052516Z Gathering data for ns/openshift-network-operator...
2026-09-16T19:48:57.462268771Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:48:57.473883764Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T19:48:57.650255861Z Collecting kubelet logs for node ip-10-0-137-205.ec2.internal
2026-09-16T19:48:57.651908719Z Collecting kubelet logs for node ip-10-0-141-132.ec2.internal
2026-09-16T19:48:57.653528007Z Collecting kubelet logs for node ip-10-0-133-109.ec2.internal
2026-09-16T19:48:57.861826721Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T19:48:58.015378780Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-gfl5f namespace
2026-09-16T19:48:58.023282642Z INFO: Node performance data collection complete.
2026-09-16T19:48:58.562336977Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T19:48:58.935027047Z Gathering data for ns/openshift-apiserver...
2026-09-16T19:48:59.434784533Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T19:48:59.741949377Z Gathering data for ns/openshift-controller-manager...
2026-09-16T19:49:00.103030112Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T19:49:00.833367208Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T19:49:01.228803517Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T19:49:01.652143004Z Gathering data for ns/openshift-service-ca...
2026-09-16T19:49:02.091532049Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T19:49:02.466761236Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-16T19:49:03.359524666Z Wrote inspect data to must-gather.
2026-09-16T19:49:03.359569377Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:49:03.359569377Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T19:49:03.492343707Z error: the server doesn't have a resource type "clusters"
2026-09-16T19:49:03.622377147Z Caches written to disk