2026-09-19T03:05:17.446954244Z [disk usage checker] Started
2026-09-19T03:05:17.450366518Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-19T03:05:17.644583264Z Gathering data for ns/openshift-cluster-version...
2026-09-19T03:05:17.777076933Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T03:05:17.857471007Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:05:17.940217996Z Gathering data for ns/default...
2026-09-19T03:05:18.249731476Z Gathering data for ns/openshift...
2026-09-19T03:05:18.683957807Z Gathering data for ns/kube-system...
2026-09-19T03:05:19.334115830Z Gathering data for ns/openshift-etcd...
2026-09-19T03:05:19.625648481Z Wrote inspect data to must-gather.
2026-09-19T03:05:19.625679317Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T03:05:19.625679317Z     namespaces "assisted-installer" not found
2026-09-19T03:05:19.984557615Z Waiting on subprocesses to finish execution.
2026-09-19T03:05:19.994369766Z Executing Istio gather script
2026-09-19T03:05:19.998327230Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-19T03:05:20.000017203Z INFO: Gathering HAProxy config files
2026-09-19T03:05:20.001028434Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-19T03:05:20.006562972Z INFO: Collecting host service logs for crio
2026-09-19T03:05:20.006837550Z INFO: Collecting host service logs for kubelet
2026-09-19T03:05:20.008257891Z INFO: Collecting host service logs for rpm-ostreed
2026-09-19T03:05:20.008559734Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-19T03:05:20.008854909Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-19T03:05:20.009165110Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-19T03:05:20.009459861Z INFO: Collecting host service logs for NetworkManager
2026-09-19T03:05:20.009740424Z INFO: Collecting host service logs for openvswitch
2026-09-19T03:05:20.010028202Z INFO: Collecting host service logs for ovs-configuration
2026-09-19T03:05:20.010337939Z INFO: Collecting host service logs for ovsdb-server
2026-09-19T03:05:20.010610572Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-19T03:05:20.010944586Z INFO: Waiting for worker host service log collection to complete ...
2026-09-19T03:05:20.013004782Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-19T03:05:20.051585357Z INFO: Waiting for node performance related collection to complete ...
2026-09-19T03:05:20.534029284Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-19T03:05:20.824720402Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-19T03:05:20.923767505Z INFO: "metallb-operator" not detected. Skipping.
2026-09-19T03:05:21.011501471Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-19T03:05:21.145664051Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-19T03:05:21.145704432Z INFO: on-disk MachineConfig config collection complete.
2026-09-19T03:05:21.226162009Z ERROR: No running kube-apiserver pods found
2026-09-19T03:05:21.226887307Z No resources found
2026-09-19T03:05:21.242431684Z error: the server doesn't have a resource type "performanceprofile"
2026-09-19T03:05:21.256853288Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-19T03:05:21.355931707Z No resources found in openshift-etcd namespace.
2026-09-19T03:05:21.362022756Z Wrote inspect data to must-gather.
2026-09-19T03:05:21.461450727Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-vww7w
2026-09-19T03:05:21.466742511Z insights-runtime-extractor-6wsrq
2026-09-19T03:05:21.466817517Z insights-runtime-extractor-j6pkx
2026-09-19T03:05:21.466949605Z insights-runtime-extractor-tbwxd
2026-09-19T03:05:21.676182810Z INFO: Worker host service log collection to complete.
2026-09-19T03:05:21.804652582Z No resources found
2026-09-19T03:05:21.820898115Z Wrote inspect data to must-gather.
2026-09-19T03:05:21.971687513Z Wrote inspect data to must-gather.
2026-09-19T03:05:21.981532919Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-19T03:05:22.211695801Z Wrote inspect data to must-gather.
2026-09-19T03:05:22.277514031Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-19T03:05:22.277514031Z See 'oc rsync -h' for help and examples
2026-09-19T03:05:22.278236229Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-19T03:05:22.289290294Z Wrote inspect data to must-gather.
2026-09-19T03:05:22.354125893Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-19T03:05:22.429886587Z Wrote inspect data to must-gather.
2026-09-19T03:05:22.443802137Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:05:22.474505340Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-19T03:05:22.475674940Z INFO: 'previous-logs' folder not found on ip-10-0-134-146.ec2.internal, skipping...
2026-09-19T03:05:22.476859255Z Gathering data for ns/openshift-monitoring...
2026-09-19T03:05:22.542770906Z Wrote inspect data to must-gather.
2026-09-19T03:05:22.604340730Z tar: Removing leading `/' from member names
2026-09-19T03:05:22.616953987Z INFO: HAProxy config collection complete.
2026-09-19T03:05:22.632082993Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-19T03:05:22.662969687Z Wrote inspect data to must-gather.
2026-09-19T03:05:22.681923897Z error: the server doesn't have a resource type "machineconfigs"
2026-09-19T03:05:22.779318562Z INFO: 'previous-logs' folder not found on ip-10-0-137-220.ec2.internal, skipping...
2026-09-19T03:05:22.853969471Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-19T03:05:22.923250586Z INFO: Getting rules from prometheus-k8s-0
2026-09-19T03:05:22.949427225Z Wrote inspect data to must-gather.
2026-09-19T03:05:23.058645900Z INFO: 'previous-logs' folder not found on ip-10-0-137-27.ec2.internal, skipping...
2026-09-19T03:05:23.058689955Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-19T03:05:23.058716624Z INFO: Machine Config Daemon termination log collection complete.
2026-09-19T03:05:23.096645312Z Wrote inspect data to must-gather.
2026-09-19T03:05:23.191400008Z Wrote inspect data to must-gather.
2026-09-19T03:05:23.248162423Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-19T03:05:23.316265692Z INFO: Getting status/config from prometheus-k8s-0
2026-09-19T03:05:23.475585229Z Wrote inspect data to must-gather.
2026-09-19T03:05:23.476342015Z INFO: INTERCONNECT MODE
2026-09-19T03:05:23.476372048Z INFO: Gathering ovn-kubernetes DBs
2026-09-19T03:05:23.621557878Z INFO: Gathering OVN_Northbound from ovnkube-node-c8wjm...
2026-09-19T03:05:23.621764459Z INFO: Gathering OVN_Northbound from ovnkube-node-gsmzv...
2026-09-19T03:05:23.621961237Z INFO: Gathering OVN_Northbound from ovnkube-node-ncqjp...
2026-09-19T03:05:23.622260294Z INFO: Gathering OVN_Southbound from ovnkube-node-c8wjm...
2026-09-19T03:05:23.622502108Z INFO: Gathering OVN_Southbound from ovnkube-node-gsmzv...
2026-09-19T03:05:23.622808974Z INFO: Gathering OVN_Southbound from ovnkube-node-ncqjp...
2026-09-19T03:05:23.698447610Z Wrote inspect data to must-gather.
2026-09-19T03:05:23.830224744Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-19T03:05:23.832258766Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-19T03:05:23.944270038Z tar: Removing leading `/' from member names
2026-09-19T03:05:23.956576960Z tar: Removing leading `/' from member names
2026-09-19T03:05:23.971628530Z tar: Removing leading `/' from member names
2026-09-19T03:05:23.985711178Z tar: Removing leading `/' from member names
2026-09-19T03:05:23.993691286Z tar: Removing leading `/' from member names
2026-09-19T03:05:24.065240429Z tar: Removing leading `/' from member names
2026-09-19T03:05:24.073305766Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T03:05:24.090780625Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:24.091413310Z Wrote inspect data to must-gather.
2026-09-19T03:05:24.120723091Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-19T03:05:24.344526586Z Wrote inspect data to must-gather.
2026-09-19T03:05:24.358765404Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:24.361712145Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-19T03:05:24.381182004Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-19T03:05:24.450054079Z INFO: Waiting for network log collection to complete ...
2026-09-19T03:05:24.450198303Z INFO: Waiting for ovnk database copies to complete ...
2026-09-19T03:05:24.450724339Z INFO: Copying ovnk databases complete.
2026-09-19T03:05:24.452661467Z 3.1M	must-gather/network_logs/ovnk_database_store
2026-09-19T03:05:24.455348934Z ovnk_database_store/
2026-09-19T03:05:24.455400634Z ovnk_database_store/ovnkube-node-ncqjp_sbdb
2026-09-19T03:05:24.476449405Z ovnk_database_store/ovnkube-node-c8wjm_nbdb
2026-09-19T03:05:24.486196444Z ovnk_database_store/ovnkube-node-gsmzv_nbdb
2026-09-19T03:05:24.495675364Z ovnk_database_store/ovnkube-node-c8wjm_sbdb
2026-09-19T03:05:24.513054583Z ovnk_database_store/ovnkube-node-ncqjp_nbdb
2026-09-19T03:05:24.522492566Z ovnk_database_store/ovnkube-node-gsmzv_sbdb
2026-09-19T03:05:24.546924764Z INFO: Network log collection complete.
2026-09-19T03:05:24.571494872Z Wrote inspect data to must-gather.
2026-09-19T03:05:24.578388218Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:24.636599399Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-19T03:05:24.736484532Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:05:24.796408722Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:24.799478720Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-19T03:05:24.849528857Z Wrote inspect data to must-gather.
2026-09-19T03:05:24.914780164Z Gathering data for ns/openshift-network-console...
2026-09-19T03:05:25.006602433Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:25.041284536Z INFO: Getting status from alertmanager-main-0
2026-09-19T03:05:25.171999398Z Wrote inspect data to must-gather.
2026-09-19T03:05:25.203710865Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:25.206761572Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-19T03:05:25.436866278Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-19T03:05:25.438574339Z INFO: Fallback to identify the container image from release info
2026-09-19T03:05:25.438931164Z Gathering data for ns/openshift-console-operator...
2026-09-19T03:05:25.441022717Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:25.636720183Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:25.639117782Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-19T03:05:25.801765819Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:25.807861910Z Gathering data for ns/openshift-console...
2026-09-19T03:05:25.987518482Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:05:26.003013317Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-19T03:05:26.139773399Z daemonset.apps/perf-node-gather-daemonset created
2026-09-19T03:05:26.276398911Z Waiting for performance profile collector pods to become ready: 1
2026-09-19T03:05:26.349879600Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-19T03:05:26.687632542Z Done executing Istio gather script
2026-09-19T03:05:26.842385991Z Gathering data for ns/openshift-dns-operator...
2026-09-19T03:05:27.136296532Z Gathering data for ns/openshift-dns...
2026-09-19T03:05:27.386832432Z Waiting for performance profile collector pods to become ready: 2
2026-09-19T03:05:27.487645076Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-19T03:05:27.674608037Z Gathering data for ns/openshift-image-registry...
2026-09-19T03:05:28.097202834Z Gathering data for ns/openshift-ingress-operator...
2026-09-19T03:05:28.388033564Z Gathering data for ns/openshift-ingress...
2026-09-19T03:05:28.507923355Z Waiting for performance profile collector pods to become ready: 3
2026-09-19T03:05:28.698685908Z Gathering data for ns/openshift-ingress-canary...
2026-09-19T03:05:29.049085350Z Gathering data for ns/openshift-insights...
2026-09-19T03:05:29.627964038Z Waiting for performance profile collector pods to become ready: 4
2026-09-19T03:05:30.736362601Z Waiting for performance profile collector pods to become ready: 5
2026-09-19T03:05:30.781310145Z Gathering data for ns/openshift-jobset-operator...
2026-09-19T03:05:31.559973459Z Gathering data for ns/openshift-config...
2026-09-19T03:05:31.797199549Z Gathering data for ns/openshift-config-managed...
2026-09-19T03:05:31.861967971Z Waiting for performance profile collector pods to become ready: 6
2026-09-19T03:05:32.200996008Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-19T03:05:32.478951644Z Gathering data for ns/openshift-kube-apiserver...
2026-09-19T03:05:32.493161035Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-19T03:05:32.726438281Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-19T03:05:32.984028711Z Waiting for performance profile collector pods to become ready: 7
2026-09-19T03:05:33.005800648Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-19T03:05:33.306967807Z Gathering data for ns/openshift-kube-scheduler...
2026-09-19T03:05:33.547805266Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-19T03:05:33.803652475Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-19T03:05:34.108376470Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-19T03:05:34.113349200Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-19T03:05:34.350085778Z Collecting performance related data for node ip-10-0-137-27.ec2.internal
2026-09-19T03:05:34.358695169Z Collecting performance related data for node ip-10-0-134-146.ec2.internal
2026-09-19T03:05:34.368519918Z Collecting performance related data for node ip-10-0-137-220.ec2.internal
2026-09-19T03:05:34.445165069Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-19T03:05:35.034163420Z Gathering data for ns/openshift-multus...
2026-09-19T03:05:36.389800362Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-19T03:05:37.500386850Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-19T03:05:37.562327647Z Gathering data for ns/openshift-host-network...
2026-09-19T03:05:37.844028741Z Gathering data for ns/openshift-network-diagnostics...
2026-09-19T03:05:38.342498770Z Gathering data for ns/openshift-network-node-identity...
2026-09-19T03:05:38.767409504Z Gathering data for ns/openshift-network-operator...
2026-09-19T03:05:39.116890418Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-19T03:05:39.368027366Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-19T03:05:39.820773850Z Gathering data for ns/openshift-apiserver-operator...
2026-09-19T03:05:40.100965011Z Gathering data for ns/openshift-apiserver...
2026-09-19T03:05:40.463667382Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-19T03:05:40.702113578Z Gathering data for ns/openshift-controller-manager...
2026-09-19T03:05:40.965291483Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-19T03:05:41.595146292Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-19T03:05:41.854120804Z Gathering data for ns/openshift-service-ca-operator...
2026-09-19T03:05:42.144007513Z Gathering data for ns/openshift-service-ca...
2026-09-19T03:05:42.346643345Z Collecting kubelet logs for node ip-10-0-137-27.ec2.internal
2026-09-19T03:05:42.350547918Z Collecting kubelet logs for node ip-10-0-134-146.ec2.internal
2026-09-19T03:05:42.352679291Z Collecting kubelet logs for node ip-10-0-137-220.ec2.internal
2026-09-19T03:05:42.506035388Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-09-19T03:05:42.515276105Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-19T03:05:42.670378478Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-6n98x namespace
2026-09-19T03:05:42.675194800Z INFO: Node performance data collection complete.
2026-09-19T03:05:43.371302328Z Wrote inspect data to must-gather.
2026-09-19T03:05:43.371343802Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T03:05:43.371343802Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-19T03:05:43.503276237Z error: the server doesn't have a resource type "clusters"
2026-09-19T03:05:43.631123453Z Caches written to disk