2026-07-31T13:03:31.491444528Z [disk usage checker] Started
2026-07-31T13:03:31.494763174Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-31T13:03:31.716639417Z Gathering data for ns/openshift-cluster-version...
2026-07-31T13:03:31.873744808Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T13:03:31.976297098Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T13:03:32.074704293Z Gathering data for ns/default...
2026-07-31T13:03:32.396910349Z Gathering data for ns/openshift...
2026-07-31T13:03:32.841486198Z Gathering data for ns/kube-system...
2026-07-31T13:03:33.576958781Z Gathering data for ns/openshift-etcd...
2026-07-31T13:03:33.906464450Z Wrote inspect data to must-gather.
2026-07-31T13:03:33.906501558Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T13:03:33.906501558Z     namespaces "assisted-installer" not found
2026-07-31T13:03:34.256964713Z Waiting on subprocesses to finish execution.
2026-07-31T13:03:34.266953761Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-31T13:03:34.269162988Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-31T13:03:34.270941753Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-31T13:03:34.277469270Z Executing Istio gather script
2026-07-31T13:03:34.281081465Z INFO: Gathering HAProxy config files
2026-07-31T13:03:34.308477499Z INFO: Collecting host service logs for crio
2026-07-31T13:03:34.308844265Z INFO: Collecting host service logs for kubelet
2026-07-31T13:03:34.309163894Z INFO: Collecting host service logs for rpm-ostreed
2026-07-31T13:03:34.309472047Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-31T13:03:34.309762391Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-31T13:03:34.310072612Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-31T13:03:34.310353127Z INFO: Collecting host service logs for NetworkManager
2026-07-31T13:03:34.310651367Z INFO: Collecting host service logs for openvswitch
2026-07-31T13:03:34.311003190Z INFO: Collecting host service logs for ovs-configuration
2026-07-31T13:03:34.311311075Z INFO: Collecting host service logs for ovsdb-server
2026-07-31T13:03:34.311647023Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-31T13:03:34.312103741Z INFO: Waiting for worker host service log collection to complete ...
2026-07-31T13:03:34.533506293Z INFO: Waiting for node performance related collection to complete ...
2026-07-31T13:03:35.080756777Z No resources found
2026-07-31T13:03:35.099773787Z INFO: "metallb-operator" not detected. Skipping.
2026-07-31T13:03:35.108695895Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-31T13:03:35.113563710Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-31T13:03:35.137798931Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-31T13:03:35.137966317Z INFO: on-disk MachineConfig config collection complete.
2026-07-31T13:03:35.145437153Z No resources found in openshift-etcd namespace.
2026-07-31T13:03:35.157630019Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-31T13:03:35.160888466Z INFO: Collecting Insights Archives from insights-operator-77696685b-bbfh7
2026-07-31T13:03:35.160938498Z insights-runtime-extractor-dj527
2026-07-31T13:03:35.160957144Z insights-runtime-extractor-frpvj
2026-07-31T13:03:35.160992046Z insights-runtime-extractor-mll8g
2026-07-31T13:03:35.438849812Z error: the server doesn't have a resource type "performanceprofile"
2026-07-31T13:03:35.463061974Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-31T13:03:35.767052208Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-31T13:03:35.767052208Z See 'oc rsync -h' for help and examples
2026-07-31T13:03:35.806229566Z Wrote inspect data to must-gather.
2026-07-31T13:03:35.871453609Z ERROR: No running kube-apiserver pods found
2026-07-31T13:03:36.168072053Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-31T13:03:36.295593210Z No resources found
2026-07-31T13:03:36.403930865Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-31T13:03:36.408553557Z Wrote inspect data to must-gather.
2026-07-31T13:03:36.486451643Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T13:03:36.542943291Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-31T13:03:36.565521099Z INFO: Worker host service log collection to complete.
2026-07-31T13:03:36.692040515Z Wrote inspect data to must-gather.
2026-07-31T13:03:36.775027682Z Wrote inspect data to must-gather.
2026-07-31T13:03:36.855406341Z Wrote inspect data to must-gather.
2026-07-31T13:03:36.881263076Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-31T13:03:36.934061607Z Wrote inspect data to must-gather.
2026-07-31T13:03:36.952076851Z tar: Removing leading `/' from member names
2026-07-31T13:03:36.963298007Z INFO: HAProxy config collection complete.
2026-07-31T13:03:36.965702735Z INFO: 'previous-logs' folder not found on ip-10-0-131-255.ec2.internal, skipping...
2026-07-31T13:03:37.049490848Z Wrote inspect data to must-gather.
2026-07-31T13:03:37.112199745Z error: the server doesn't have a resource type "machineconfigs"
2026-07-31T13:03:37.140257508Z Gathering data for ns/openshift-monitoring...
2026-07-31T13:03:37.184970003Z Wrote inspect data to must-gather.
2026-07-31T13:03:37.189166071Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-31T13:03:37.304578788Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-31T13:03:37.312372976Z INFO: 'previous-logs' folder not found on ip-10-0-133-71.ec2.internal, skipping...
2026-07-31T13:03:37.396157026Z INFO: Getting rules from prometheus-k8s-0
2026-07-31T13:03:37.478257367Z Wrote inspect data to must-gather.
2026-07-31T13:03:37.564504455Z Wrote inspect data to must-gather.
2026-07-31T13:03:37.596878097Z INFO: 'previous-logs' folder not found on ip-10-0-142-95.ec2.internal, skipping...
2026-07-31T13:03:37.596923269Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-31T13:03:37.596923269Z INFO: Machine Config Daemon termination log collection complete.
2026-07-31T13:03:37.734905959Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-31T13:03:37.755107874Z Wrote inspect data to must-gather.
2026-07-31T13:03:37.848788328Z INFO: Getting status/config from prometheus-k8s-0
2026-07-31T13:03:37.961369917Z Wrote inspect data to must-gather.
2026-07-31T13:03:38.049143211Z INFO: INTERCONNECT MODE
2026-07-31T13:03:38.049189399Z INFO: Gathering ovn-kubernetes DBs
2026-07-31T13:03:38.169484820Z Wrote inspect data to must-gather.
2026-07-31T13:03:38.222486172Z INFO: Gathering OVN_Northbound from ovnkube-node-47d66...
2026-07-31T13:03:38.222754901Z INFO: Gathering OVN_Northbound from ovnkube-node-4bjbc...
2026-07-31T13:03:38.223014249Z INFO: Gathering OVN_Northbound from ovnkube-node-zfh8z...
2026-07-31T13:03:38.223274456Z INFO: Gathering OVN_Southbound from ovnkube-node-47d66...
2026-07-31T13:03:38.223519803Z INFO: Gathering OVN_Southbound from ovnkube-node-4bjbc...
2026-07-31T13:03:38.223775682Z INFO: Gathering OVN_Southbound from ovnkube-node-zfh8z...
2026-07-31T13:03:38.294956589Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-31T13:03:38.315918624Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-31T13:03:38.565174228Z Wrote inspect data to must-gather.
2026-07-31T13:03:38.572097368Z tar: Removing leading `/' from member names
2026-07-31T13:03:38.582882985Z tar: Removing leading `/' from member names
2026-07-31T13:03:38.587835250Z tar: Removing leading `/' from member names
2026-07-31T13:03:38.599673638Z tar: Removing leading `/' from member names
2026-07-31T13:03:38.600120695Z tar: Removing leading `/' from member names
2026-07-31T13:03:38.606778897Z tar: Removing leading `/' from member names
2026-07-31T13:03:38.643761301Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:38.698003630Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-31T13:03:38.847000498Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T13:03:38.849043186Z Wrote inspect data to must-gather.
2026-07-31T13:03:38.912295070Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:38.916518951Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-31T13:03:38.988882546Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-31T13:03:39.093314375Z Wrote inspect data to must-gather.
2026-07-31T13:03:39.128088914Z INFO: Waiting for network log collection to complete ...
2026-07-31T13:03:39.128222222Z INFO: Waiting for ovnk database copies to complete ...
2026-07-31T13:03:39.128696781Z INFO: Copying ovnk databases complete.
2026-07-31T13:03:39.131016616Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-31T13:03:39.134380397Z ovnk_database_store/
2026-07-31T13:03:39.134440653Z ovnk_database_store/ovnkube-node-4bjbc_sbdb
2026-07-31T13:03:39.153474944Z ovnk_database_store/ovnkube-node-zfh8z_nbdb
2026-07-31T13:03:39.159960265Z ovnk_database_store/ovnkube-node-47d66_sbdb
2026-07-31T13:03:39.167692237Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:39.174057530Z ovnk_database_store/ovnkube-node-4bjbc_nbdb
2026-07-31T13:03:39.179672989Z ovnk_database_store/ovnkube-node-zfh8z_sbdb
2026-07-31T13:03:39.197042292Z ovnk_database_store/ovnkube-node-47d66_nbdb
2026-07-31T13:03:39.208236192Z INFO: Network log collection complete.
2026-07-31T13:03:39.275700738Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-31T13:03:39.351107112Z Wrote inspect data to must-gather.
2026-07-31T13:03:39.395249020Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:39.398657335Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-31T13:03:39.476603558Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T13:03:39.596827088Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:39.664472480Z Wrote inspect data to must-gather.
2026-07-31T13:03:39.677706742Z Gathering data for ns/openshift-network-console...
2026-07-31T13:03:39.694243701Z INFO: Getting status from alertmanager-main-0
2026-07-31T13:03:39.836652461Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:39.839226298Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-31T13:03:39.969207249Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-31T13:03:39.972338655Z INFO: Fallback to identify the container image from release info
2026-07-31T13:03:40.095689289Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:40.166945408Z Gathering data for ns/openshift-console-operator...
2026-07-31T13:03:40.305225336Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:40.307469681Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-31T13:03:40.482235680Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:40.572316095Z Gathering data for ns/openshift-console...
2026-07-31T13:03:40.635506296Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-07-31T13:03:40.700528613Z Wrote inspect data to /must-gather/istio.
2026-07-31T13:03:40.778974125Z daemonset.apps/perf-node-gather-daemonset created
2026-07-31T13:03:40.917690091Z Waiting for performance profile collector pods to become ready: 1
2026-07-31T13:03:41.166213030Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-31T13:03:41.430904909Z Done executing Istio gather script
2026-07-31T13:03:41.549355494Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-31T13:03:41.737035297Z Gathering data for ns/openshift-dns-operator...
2026-07-31T13:03:42.046062818Z Waiting for performance profile collector pods to become ready: 2
2026-07-31T13:03:42.071952673Z Gathering data for ns/openshift-dns...
2026-07-31T13:03:42.701759215Z Gathering data for ns/openshift-image-registry...
2026-07-31T13:03:43.171320785Z Waiting for performance profile collector pods to become ready: 3
2026-07-31T13:03:43.183771431Z Gathering data for ns/openshift-ingress-operator...
2026-07-31T13:03:43.516829653Z Gathering data for ns/openshift-ingress...
2026-07-31T13:03:43.874607978Z Gathering data for ns/openshift-ingress-canary...
2026-07-31T13:03:44.290502891Z Gathering data for ns/openshift-insights...
2026-07-31T13:03:44.300236139Z Waiting for performance profile collector pods to become ready: 4
2026-07-31T13:03:45.409243070Z Waiting for performance profile collector pods to become ready: 5
2026-07-31T13:03:46.211369821Z Gathering data for ns/openshift-jobset-operator...
2026-07-31T13:03:46.541665049Z Waiting for performance profile collector pods to become ready: 6
2026-07-31T13:03:46.555160419Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-31T13:03:47.062128607Z Gathering data for ns/openshift-config...
2026-07-31T13:03:47.389772912Z Gathering data for ns/openshift-config-managed...
2026-07-31T13:03:47.662841443Z Waiting for performance profile collector pods to become ready: 7
2026-07-31T13:03:47.879864961Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-31T13:03:48.156280392Z Gathering data for ns/openshift-kube-apiserver...
2026-07-31T13:03:48.469029352Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-31T13:03:48.770473633Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-31T13:03:48.796727901Z Waiting for performance profile collector pods to become ready: 8
2026-07-31T13:03:49.077423198Z Gathering data for ns/openshift-kube-scheduler...
2026-07-31T13:03:49.341437182Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-31T13:03:49.630579285Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-31T13:03:49.919598730Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-31T13:03:49.983896877Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-31T13:03:50.210902172Z Collecting performance related data for node ip-10-0-142-95.ec2.internal
2026-07-31T13:03:50.218431717Z Collecting performance related data for node ip-10-0-133-71.ec2.internal
2026-07-31T13:03:50.225514109Z Collecting performance related data for node ip-10-0-131-255.ec2.internal
2026-07-31T13:03:50.370954932Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-31T13:03:50.998538524Z Gathering data for ns/openshift-multus...
2026-07-31T13:03:51.560882647Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-31T13:03:52.594488745Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-31T13:03:53.956181348Z Gathering data for ns/openshift-host-network...
2026-07-31T13:03:54.279885198Z Gathering data for ns/openshift-network-diagnostics...
2026-07-31T13:03:54.858130202Z Gathering data for ns/openshift-network-node-identity...
2026-07-31T13:03:55.333485338Z Gathering data for ns/openshift-network-operator...
2026-07-31T13:03:55.759002920Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-31T13:03:56.058013142Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-31T13:03:56.565736052Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-07-31T13:03:56.591056351Z Gathering data for ns/openshift-apiserver-operator...
2026-07-31T13:03:56.952527710Z Gathering data for ns/openshift-apiserver...
2026-07-31T13:03:57.358869183Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-31T13:03:57.653922136Z Gathering data for ns/openshift-controller-manager...
2026-07-31T13:03:57.978897494Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-31T13:03:58.705084610Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-31T13:03:58.844080656Z Collecting kubelet logs for node ip-10-0-142-95.ec2.internal
2026-07-31T13:03:58.846159334Z Collecting kubelet logs for node ip-10-0-133-71.ec2.internal
2026-07-31T13:03:58.849215348Z Collecting kubelet logs for node ip-10-0-131-255.ec2.internal
2026-07-31T13:03:59.057699458Z Gathering data for ns/openshift-service-ca-operator...
2026-07-31T13:03:59.207896370Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-5d2np namespace
2026-07-31T13:03:59.212920160Z INFO: Node performance data collection complete.
2026-07-31T13:03:59.441682445Z Gathering data for ns/openshift-service-ca...
2026-07-31T13:03:59.818531666Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-31T13:04:00.845556919Z Wrote inspect data to must-gather.
2026-07-31T13:04:00.845607449Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T13:04:00.845607449Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-31T13:04:00.979595221Z error: the server doesn't have a resource type "clusters"
2026-07-31T13:04:01.108801380Z Caches written to disk