2026-07-30T07:18:24.859249330Z [disk usage checker] Started
2026-07-30T07:18:24.863295299Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T07:18:25.067583478Z Gathering data for ns/openshift-cluster-version...
2026-07-30T07:18:25.198314697Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T07:18:25.289590720Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T07:18:25.377635749Z Gathering data for ns/default...
2026-07-30T07:18:25.675724522Z Gathering data for ns/openshift...
2026-07-30T07:18:26.077416752Z Gathering data for ns/kube-system...
2026-07-30T07:18:26.832909843Z Gathering data for ns/openshift-etcd...
2026-07-30T07:18:27.261125374Z Wrote inspect data to must-gather.
2026-07-30T07:18:27.261157477Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T07:18:27.261157477Z     namespaces "assisted-installer" not found
2026-07-30T07:18:27.520207108Z Waiting on subprocesses to finish execution.
2026-07-30T07:18:27.528644102Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-30T07:18:27.531056999Z INFO: Gathering HAProxy config files
2026-07-30T07:18:27.534963014Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-30T07:18:27.535594529Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-30T07:18:27.537284302Z Executing Istio gather script
2026-07-30T07:18:27.540621028Z INFO: Collecting host service logs for crio
2026-07-30T07:18:27.541052624Z INFO: Collecting host service logs for kubelet
2026-07-30T07:18:27.541402058Z INFO: Collecting host service logs for rpm-ostreed
2026-07-30T07:18:27.541815205Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-30T07:18:27.542161862Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-30T07:18:27.542491940Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-30T07:18:27.542860735Z INFO: Collecting host service logs for NetworkManager
2026-07-30T07:18:27.543193276Z INFO: Collecting host service logs for openvswitch
2026-07-30T07:18:27.543546406Z INFO: Collecting host service logs for ovs-configuration
2026-07-30T07:18:27.543925221Z INFO: Collecting host service logs for ovsdb-server
2026-07-30T07:18:27.545012428Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-30T07:18:27.545465886Z INFO: Waiting for worker host service log collection to complete ...
2026-07-30T07:18:28.044239709Z INFO: "metallb-operator" not detected. Skipping.
2026-07-30T07:18:28.059329797Z No resources found
2026-07-30T07:18:28.238725553Z INFO: Waiting for node performance related collection to complete ...
2026-07-30T07:18:28.253230280Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-30T07:18:28.342173904Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-30T07:18:28.582113609Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-30T07:18:28.582147180Z INFO: on-disk MachineConfig config collection complete.
2026-07-30T07:18:28.648606523Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-sflbp
2026-07-30T07:18:28.648670337Z insights-runtime-extractor-2g9wh
2026-07-30T07:18:28.648695731Z insights-runtime-extractor-7jxg6
2026-07-30T07:18:28.648716500Z insights-runtime-extractor-cq6j9
2026-07-30T07:18:28.671772862Z Wrote inspect data to must-gather.
2026-07-30T07:18:28.746926869Z No resources found in openshift-etcd namespace.
2026-07-30T07:18:28.791557450Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-30T07:18:29.012018742Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-30T07:18:29.017955038Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-30T07:18:29.068764703Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-30T07:18:29.068764703Z See 'oc rsync -h' for help and examples
2026-07-30T07:18:29.077831350Z INFO: Worker host service log collection to complete.
2026-07-30T07:18:29.123825598Z ERROR: No running kube-apiserver pods found
2026-07-30T07:18:29.188029908Z Wrote inspect data to must-gather.
2026-07-30T07:18:29.245284759Z error: the server doesn't have a resource type "performanceprofile"
2026-07-30T07:18:29.631009765Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-30T07:18:29.786757728Z Gathering data for ns/openshift-monitoring...
2026-07-30T07:18:29.788076637Z Wrote inspect data to must-gather.
2026-07-30T07:18:29.864891784Z INFO: 'previous-logs' folder not found on ip-10-0-130-188.ec2.internal, skipping...
2026-07-30T07:18:29.888373233Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T07:18:29.911861611Z tar: Removing leading `/' from member names
2026-07-30T07:18:29.917605829Z INFO: HAProxy config collection complete.
2026-07-30T07:18:29.944050960Z Wrote inspect data to must-gather.
2026-07-30T07:18:29.994972665Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T07:18:30.105533033Z Wrote inspect data to must-gather.
2026-07-30T07:18:30.127025565Z No resources found
2026-07-30T07:18:30.135043761Z Wrote inspect data to must-gather.
2026-07-30T07:18:30.140095135Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-30T07:18:30.167981840Z Wrote inspect data to must-gather.
2026-07-30T07:18:30.197382912Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-30T07:18:30.319702027Z error: the server doesn't have a resource type "machineconfigs"
2026-07-30T07:18:30.355959710Z INFO: 'previous-logs' folder not found on ip-10-0-143-185.ec2.internal, skipping...
2026-07-30T07:18:30.443592509Z Wrote inspect data to must-gather.
2026-07-30T07:18:30.491640800Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-30T07:18:30.580291428Z INFO: Getting rules from prometheus-k8s-0
2026-07-30T07:18:30.630115286Z INFO: 'previous-logs' folder not found on ip-10-0-143-193.ec2.internal, skipping...
2026-07-30T07:18:30.630153428Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-30T07:18:30.630153428Z INFO: Machine Config Daemon termination log collection complete.
2026-07-30T07:18:30.715453447Z Wrote inspect data to must-gather.
2026-07-30T07:18:30.722034093Z Wrote inspect data to must-gather.
2026-07-30T07:18:30.876864314Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-30T07:18:30.966555469Z Wrote inspect data to must-gather.
2026-07-30T07:18:30.981379400Z INFO: Getting status/config from prometheus-k8s-0
2026-07-30T07:18:31.106239091Z Wrote inspect data to must-gather.
2026-07-30T07:18:31.227355039Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-30T07:18:31.254591964Z INFO: INTERCONNECT MODE
2026-07-30T07:18:31.254627987Z INFO: Gathering ovn-kubernetes DBs
2026-07-30T07:18:31.321893307Z Wrote inspect data to must-gather.
2026-07-30T07:18:31.420022134Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:31.433386548Z INFO: Gathering OVN_Northbound from ovnkube-node-84tq9...
2026-07-30T07:18:31.433741028Z INFO: Gathering OVN_Northbound from ovnkube-node-9ghbd...
2026-07-30T07:18:31.434025638Z INFO: Gathering OVN_Northbound from ovnkube-node-kkx25...
2026-07-30T07:18:31.434342784Z INFO: Gathering OVN_Southbound from ovnkube-node-84tq9...
2026-07-30T07:18:31.434635273Z INFO: Gathering OVN_Southbound from ovnkube-node-9ghbd...
2026-07-30T07:18:31.434977623Z INFO: Gathering OVN_Southbound from ovnkube-node-kkx25...
2026-07-30T07:18:31.480903643Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-30T07:18:31.611691316Z Wrote inspect data to must-gather.
2026-07-30T07:18:31.785911098Z tar: Removing leading `/' from member names
2026-07-30T07:18:31.799066027Z tar: Removing leading `/' from member names
2026-07-30T07:18:31.806184224Z tar: Removing leading `/' from member names
2026-07-30T07:18:31.816265203Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:31.820358977Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-30T07:18:31.822361083Z tar: Removing leading `/' from member names
2026-07-30T07:18:31.867120794Z tar: Removing leading `/' from member names
2026-07-30T07:18:31.878469142Z tar: Removing leading `/' from member names
2026-07-30T07:18:31.881150225Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-30T07:18:31.923274165Z Wrote inspect data to must-gather.
2026-07-30T07:18:31.943610742Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T07:18:32.046617890Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:32.162904904Z Wrote inspect data to must-gather.
2026-07-30T07:18:32.186306017Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-30T07:18:32.279187343Z INFO: Waiting for network log collection to complete ...
2026-07-30T07:18:32.279229386Z INFO: Waiting for ovnk database copies to complete ...
2026-07-30T07:18:32.279771376Z INFO: Copying ovnk databases complete.
2026-07-30T07:18:32.282054324Z 3.5M	must-gather/network_logs/ovnk_database_store
2026-07-30T07:18:32.285007591Z ovnk_database_store/
2026-07-30T07:18:32.285007591Z ovnk_database_store/ovnkube-node-84tq9_sbdb
2026-07-30T07:18:32.294014666Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:32.296095862Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-30T07:18:32.312803540Z ovnk_database_store/ovnkube-node-kkx25_sbdb
2026-07-30T07:18:32.333954902Z ovnk_database_store/ovnkube-node-kkx25_nbdb
2026-07-30T07:18:32.341943441Z ovnk_database_store/ovnkube-node-84tq9_nbdb
2026-07-30T07:18:32.352421975Z ovnk_database_store/ovnkube-node-9ghbd_nbdb
2026-07-30T07:18:32.362432258Z ovnk_database_store/ovnkube-node-9ghbd_sbdb
2026-07-30T07:18:32.388056994Z INFO: Network log collection complete.
2026-07-30T07:18:32.408740589Z Wrote inspect data to must-gather.
2026-07-30T07:18:32.508098872Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-30T07:18:32.509628666Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:32.619407099Z W0730 07:18:32.619356    1539 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-30T07:18:32.628333123Z Wrote inspect data to must-gather.
2026-07-30T07:18:32.638141017Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T07:18:32.728568453Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:32.731303897Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-30T07:18:32.887762184Z Gathering data for ns/openshift-network-console...
2026-07-30T07:18:32.911048914Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-30T07:18:32.913732426Z INFO: Fallback to identify the container image from release info
2026-07-30T07:18:32.934503417Z INFO: Getting status from alertmanager-main-0
2026-07-30T07:18:32.981087764Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:33.182551683Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:33.185394700Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-30T07:18:33.349203745Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:33.387292540Z Gathering data for ns/openshift-console-operator...
2026-07-30T07:18:33.548866890Z Wrote inspect data to /must-gather/istio.
2026-07-30T07:18:33.619178766Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-30T07:18:33.770931232Z daemonset.apps/perf-node-gather-daemonset created
2026-07-30T07:18:33.899861118Z Gathering data for ns/openshift-console...
2026-07-30T07:18:33.900261797Z Waiting for performance profile collector pods to become ready: 1
2026-07-30T07:18:34.255593313Z Done executing Istio gather script
2026-07-30T07:18:34.508416747Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-30T07:18:34.914802724Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T07:18:35.023792244Z Waiting for performance profile collector pods to become ready: 2
2026-07-30T07:18:35.060520271Z Gathering data for ns/openshift-dns-operator...
2026-07-30T07:18:35.425321237Z Gathering data for ns/openshift-dns...
2026-07-30T07:18:36.143539992Z Waiting for performance profile collector pods to become ready: 3
2026-07-30T07:18:36.146989615Z Gathering data for ns/openshift-image-registry...
2026-07-30T07:18:36.635164483Z Gathering data for ns/openshift-ingress-operator...
2026-07-30T07:18:36.955568046Z Gathering data for ns/openshift-ingress...
2026-07-30T07:18:37.267038695Z Waiting for performance profile collector pods to become ready: 4
2026-07-30T07:18:37.325375178Z Gathering data for ns/openshift-ingress-canary...
2026-07-30T07:18:37.842622377Z Gathering data for ns/openshift-insights...
2026-07-30T07:18:38.387166943Z Waiting for performance profile collector pods to become ready: 5
2026-07-30T07:18:39.514113036Z Waiting for performance profile collector pods to become ready: 6
2026-07-30T07:18:39.919750535Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T07:18:40.090866410Z Gathering data for ns/openshift-config...
2026-07-30T07:18:40.400568773Z Gathering data for ns/openshift-config-managed...
2026-07-30T07:18:40.622226699Z Waiting for performance profile collector pods to become ready: 7
2026-07-30T07:18:40.894774041Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-30T07:18:41.200190486Z Gathering data for ns/openshift-kube-apiserver...
2026-07-30T07:18:41.524365204Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-30T07:18:41.739021695Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-30T07:18:41.862234288Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-30T07:18:42.000549212Z Collecting performance related data for node ip-10-0-143-193.ec2.internal
2026-07-30T07:18:42.007443806Z Collecting performance related data for node ip-10-0-130-188.ec2.internal
2026-07-30T07:18:42.015141538Z Collecting performance related data for node ip-10-0-143-185.ec2.internal
2026-07-30T07:18:42.238895163Z Gathering data for ns/openshift-kube-scheduler...
2026-07-30T07:18:42.547684531Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-30T07:18:42.955520249Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-30T07:18:43.382510309Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-30T07:18:43.753723934Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-30T07:18:44.266065620Z Gathering data for ns/openshift-multus...
2026-07-30T07:18:44.924727575Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T07:18:45.994343005Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-30T07:18:47.808087556Z Gathering data for ns/openshift-host-network...
2026-07-30T07:18:48.158750670Z Gathering data for ns/openshift-network-diagnostics...
2026-07-30T07:18:48.771563707Z Gathering data for ns/openshift-network-node-identity...
2026-07-30T07:18:49.257016891Z Gathering data for ns/openshift-network-operator...
2026-07-30T07:18:49.858221428Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-30T07:18:49.929390263Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T07:18:49.954844488Z Collecting kubelet logs for node ip-10-0-143-193.ec2.internal
2026-07-30T07:18:49.956742433Z Collecting kubelet logs for node ip-10-0-130-188.ec2.internal
2026-07-30T07:18:49.958450670Z Collecting kubelet logs for node ip-10-0-143-185.ec2.internal
2026-07-30T07:18:50.205413238Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-30T07:18:50.338235494Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-b5dgw namespace
2026-07-30T07:18:50.349777709Z INFO: Node performance data collection complete.
2026-07-30T07:18:50.747936964Z Gathering data for ns/openshift-apiserver-operator...
2026-07-30T07:18:51.067723982Z Gathering data for ns/openshift-apiserver...
2026-07-30T07:18:51.488549677Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-30T07:18:51.787460509Z Gathering data for ns/openshift-controller-manager...
2026-07-30T07:18:52.090791187Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-30T07:18:52.856846880Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-30T07:18:53.217896119Z Gathering data for ns/openshift-service-ca-operator...
2026-07-30T07:18:53.587340427Z Gathering data for ns/openshift-service-ca...
2026-07-30T07:18:53.978482974Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-30T07:18:54.934329174Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T07:18:55.029928263Z Wrote inspect data to must-gather.
2026-07-30T07:18:55.029971773Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T07:18:55.029971773Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-30T07:18:55.154680885Z error: the server doesn't have a resource type "clusters"
2026-07-30T07:18:55.290316961Z Caches written to disk