2026-09-19T03:03:36.380478745Z [disk usage checker] Started
2026-09-19T03:03:36.383659360Z [disk usage checker] Volume usage percentage: current = 49 ; allowed = 70
2026-09-19T03:03:38.558673554Z Gathering data for ns/openshift-cluster-version...
2026-09-19T03:03:38.698762335Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T03:03:38.788397798Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:03:38.871895038Z Gathering data for ns/default...
2026-09-19T03:03:39.146933770Z Gathering data for ns/openshift...
2026-09-19T03:03:39.546228976Z Gathering data for ns/kube-system...
2026-09-19T03:03:40.419394558Z Gathering data for ns/openshift-etcd...
2026-09-19T03:03:40.719772389Z Wrote inspect data to must-gather.
2026-09-19T03:03:40.719798456Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T03:03:40.719798456Z     namespaces "assisted-installer" not found
2026-09-19T03:03:40.962676224Z Waiting on subprocesses to finish execution.
2026-09-19T03:03:40.978715209Z INFO: Gathering HAProxy config files
2026-09-19T03:03:40.978827244Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-19T03:03:40.982023654Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-19T03:03:40.982375311Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-19T03:03:40.983120503Z INFO: Collecting host service logs for crio
2026-09-19T03:03:40.983520349Z INFO: Collecting host service logs for kubelet
2026-09-19T03:03:40.983893457Z INFO: Collecting host service logs for rpm-ostreed
2026-09-19T03:03:40.984453015Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-19T03:03:40.984829862Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-19T03:03:40.985191543Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-19T03:03:40.985571740Z INFO: Collecting host service logs for NetworkManager
2026-09-19T03:03:40.985935395Z INFO: Collecting host service logs for openvswitch
2026-09-19T03:03:40.986303684Z INFO: Collecting host service logs for ovs-configuration
2026-09-19T03:03:40.986674459Z INFO: Collecting host service logs for ovsdb-server
2026-09-19T03:03:40.987110868Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-19T03:03:40.987652033Z INFO: Waiting for worker host service log collection to complete ...
2026-09-19T03:03:41.041963267Z Executing Istio gather script
2026-09-19T03:03:41.090144210Z INFO: Waiting for node performance related collection to complete ...
2026-09-19T03:03:41.538203706Z [disk usage checker] Volume usage percentage: current = 49 ; allowed = 70
2026-09-19T03:03:41.792333790Z error: the server doesn't have a resource type "performanceprofile"
2026-09-19T03:03:41.897967314Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-19T03:03:42.014737494Z INFO: "metallb-operator" not detected. Skipping.
2026-09-19T03:03:42.115231668Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-19T03:03:42.122465827Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-19T03:03:42.185090725Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-19T03:03:42.186177852Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-fw25d
2026-09-19T03:03:42.186217274Z insights-runtime-extractor-6hsjq
2026-09-19T03:03:42.186236781Z insights-runtime-extractor-qv4js
2026-09-19T03:03:42.186269872Z insights-runtime-extractor-vgnnb
2026-09-19T03:03:42.199156570Z No resources found in openshift-etcd namespace.
2026-09-19T03:03:42.224466819Z No resources found
2026-09-19T03:03:42.235615343Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-19T03:03:42.269144828Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-19T03:03:42.269178170Z INFO: on-disk MachineConfig config collection complete.
2026-09-19T03:03:42.408583562Z INFO: Worker host service log collection to complete.
2026-09-19T03:03:42.434156770Z Wrote inspect data to must-gather.
2026-09-19T03:03:42.443731240Z ERROR: No running kube-apiserver pods found
2026-09-19T03:03:42.747143003Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-19T03:03:42.747143003Z See 'oc rsync -h' for help and examples
2026-09-19T03:03:42.799596357Z tar: Removing leading `/' from member names
2026-09-19T03:03:42.812378770Z INFO: HAProxy config collection complete.
2026-09-19T03:03:42.898557499Z Wrote inspect data to must-gather.
2026-09-19T03:03:42.961224541Z Wrote inspect data to must-gather.
2026-09-19T03:03:43.136176289Z Wrote inspect data to must-gather.
2026-09-19T03:03:43.252298221Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-19T03:03:43.254449769Z No resources found
2026-09-19T03:03:43.342646615Z Wrote inspect data to must-gather.
2026-09-19T03:03:43.355192279Z INFO: 'previous-logs' folder not found on ip-10-0-130-14.ec2.internal, skipping...
2026-09-19T03:03:43.358571687Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-19T03:03:43.378782246Z Wrote inspect data to must-gather.
2026-09-19T03:03:43.435300978Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:03:43.508281407Z Gathering data for ns/openshift-monitoring...
2026-09-19T03:03:43.517200205Z error: the server doesn't have a resource type "machineconfigs"
2026-09-19T03:03:43.567569792Z Wrote inspect data to must-gather.
2026-09-19T03:03:43.698216263Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-19T03:03:43.758922456Z INFO: 'previous-logs' folder not found on ip-10-0-133-156.ec2.internal, skipping...
2026-09-19T03:03:43.818400574Z INFO: Getting rules from prometheus-k8s-0
2026-09-19T03:03:43.875773792Z Wrote inspect data to must-gather.
2026-09-19T03:03:43.963526229Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-19T03:03:43.963898162Z Wrote inspect data to must-gather.
2026-09-19T03:03:44.050146962Z INFO: 'previous-logs' folder not found on ip-10-0-137-84.ec2.internal, skipping...
2026-09-19T03:03:44.050178622Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-19T03:03:44.050211628Z INFO: Machine Config Daemon termination log collection complete.
2026-09-19T03:03:44.128771941Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-19T03:03:44.148787706Z Wrote inspect data to must-gather.
2026-09-19T03:03:44.230992009Z INFO: Getting status/config from prometheus-k8s-0
2026-09-19T03:03:44.351682070Z Wrote inspect data to must-gather.
2026-09-19T03:03:44.404879394Z Wrote inspect data to must-gather.
2026-09-19T03:03:44.567462524Z Wrote inspect data to must-gather.
2026-09-19T03:03:44.623085236Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-19T03:03:44.674901739Z INFO: INTERCONNECT MODE
2026-09-19T03:03:44.674941732Z INFO: Gathering ovn-kubernetes DBs
2026-09-19T03:03:44.786184231Z Wrote inspect data to must-gather.
2026-09-19T03:03:44.817920049Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-19T03:03:44.844250178Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-19T03:03:44.860344124Z INFO: Gathering OVN_Northbound from ovnkube-node-2v9vs...
2026-09-19T03:03:44.860745509Z INFO: Gathering OVN_Northbound from ovnkube-node-6bq2q...
2026-09-19T03:03:44.861220872Z INFO: Gathering OVN_Northbound from ovnkube-node-zngw4...
2026-09-19T03:03:44.861558885Z INFO: Gathering OVN_Southbound from ovnkube-node-2v9vs...
2026-09-19T03:03:44.861870071Z INFO: Gathering OVN_Southbound from ovnkube-node-6bq2q...
2026-09-19T03:03:44.862210066Z INFO: Gathering OVN_Southbound from ovnkube-node-zngw4...
2026-09-19T03:03:44.978215100Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T03:03:45.120425883Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:45.122450268Z Wrote inspect data to must-gather.
2026-09-19T03:03:45.167939245Z tar: Removing leading `/' from member names
2026-09-19T03:03:45.200714439Z tar: Removing leading `/' from member names
2026-09-19T03:03:45.207732806Z tar: Removing leading `/' from member names
2026-09-19T03:03:45.232607513Z tar: Removing leading `/' from member names
2026-09-19T03:03:45.234665564Z tar: Removing leading `/' from member names
2026-09-19T03:03:45.285527944Z tar: Removing leading `/' from member names
2026-09-19T03:03:45.300051010Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-19T03:03:45.400666162Z Wrote inspect data to must-gather.
2026-09-19T03:03:45.411806908Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:45.416464382Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-19T03:03:45.570119255Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-19T03:03:45.642946920Z W0919 03:03:45.642906    1468 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-19T03:03:45.646739623Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:45.649631004Z Wrote inspect data to must-gather.
2026-09-19T03:03:45.719342013Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:03:45.760931906Z INFO: Waiting for network log collection to complete ...
2026-09-19T03:03:45.761457088Z INFO: Waiting for ovnk database copies to complete ...
2026-09-19T03:03:45.762058170Z INFO: Copying ovnk databases complete.
2026-09-19T03:03:45.763946824Z 3.1M	must-gather/network_logs/ovnk_database_store
2026-09-19T03:03:45.767241398Z ovnk_database_store/
2026-09-19T03:03:45.767297335Z ovnk_database_store/ovnkube-node-zngw4_sbdb
2026-09-19T03:03:45.789876824Z ovnk_database_store/ovnkube-node-6bq2q_nbdb
2026-09-19T03:03:45.799770908Z ovnk_database_store/ovnkube-node-zngw4_nbdb
2026-09-19T03:03:45.808267190Z ovnk_database_store/ovnkube-node-2v9vs_nbdb
2026-09-19T03:03:45.818524817Z ovnk_database_store/ovnkube-node-6bq2q_sbdb
2026-09-19T03:03:45.835965521Z ovnk_database_store/ovnkube-node-2v9vs_sbdb
2026-09-19T03:03:45.861942415Z INFO: Network log collection complete.
2026-09-19T03:03:45.897728230Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:45.900987768Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-19T03:03:45.937534867Z Gathering data for ns/openshift-network-console...
2026-09-19T03:03:45.984550496Z Wrote inspect data to must-gather.
2026-09-19T03:03:46.062238658Z INFO: Getting status from alertmanager-main-0
2026-09-19T03:03:46.114390380Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:46.263944351Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-19T03:03:46.267616504Z INFO: Fallback to identify the container image from release info
2026-09-19T03:03:46.337512432Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:46.340611873Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-19T03:03:46.409370930Z Gathering data for ns/openshift-console-operator...
2026-09-19T03:03:46.545129513Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:46.582576991Z [disk usage checker] Volume usage percentage: current = 49 ; allowed = 70
2026-09-19T03:03:46.734951938Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:46.738908891Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-19T03:03:46.786196850Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-19T03:03:46.816390964Z Gathering data for ns/openshift-console...
2026-09-19T03:03:46.910054679Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:46.911304062Z daemonset.apps/perf-node-gather-daemonset created
2026-09-19T03:03:47.047801763Z Waiting for performance profile collector pods to become ready: 1
2026-09-19T03:03:47.134793733Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:03:47.478504597Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-19T03:03:47.825954468Z Done executing Istio gather script
2026-09-19T03:03:47.990838823Z Gathering data for ns/openshift-dns-operator...
2026-09-19T03:03:48.151670792Z Waiting for performance profile collector pods to become ready: 2
2026-09-19T03:03:48.267016403Z Gathering data for ns/openshift-dns...
2026-09-19T03:03:48.817486625Z Gathering data for ns/openshift-image-registry...
2026-09-19T03:03:49.266500481Z Gathering data for ns/openshift-ingress-operator...
2026-09-19T03:03:49.270208353Z Waiting for performance profile collector pods to become ready: 3
2026-09-19T03:03:49.559701946Z Gathering data for ns/openshift-ingress...
2026-09-19T03:03:49.951452891Z Gathering data for ns/openshift-ingress-canary...
2026-09-19T03:03:50.337178525Z Gathering data for ns/openshift-insights...
2026-09-19T03:03:50.378783428Z Waiting for performance profile collector pods to become ready: 4
2026-09-19T03:03:51.482010470Z Waiting for performance profile collector pods to become ready: 5
2026-09-19T03:03:51.587123134Z [disk usage checker] Volume usage percentage: current = 49 ; allowed = 70
2026-09-19T03:03:52.091059235Z Gathering data for ns/openshift-jobset-operator...
2026-09-19T03:03:52.587023339Z Waiting for performance profile collector pods to become ready: 6
2026-09-19T03:03:52.907359457Z Gathering data for ns/openshift-config...
2026-09-19T03:03:53.202165013Z Gathering data for ns/openshift-config-managed...
2026-09-19T03:03:53.603553059Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-19T03:03:53.706911316Z Waiting for performance profile collector pods to become ready: 7
2026-09-19T03:03:53.890409030Z Gathering data for ns/openshift-kube-apiserver...
2026-09-19T03:03:54.151524549Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-19T03:03:54.487818321Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-19T03:03:54.821050504Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-19T03:03:54.829503176Z Gathering data for ns/openshift-kube-scheduler...
2026-09-19T03:03:55.075019705Z Collecting performance related data for node ip-10-0-133-156.ec2.internal
2026-09-19T03:03:55.084304882Z Collecting performance related data for node ip-10-0-137-84.ec2.internal
2026-09-19T03:03:55.091803822Z Collecting performance related data for node ip-10-0-130-14.ec2.internal
2026-09-19T03:03:55.143268906Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-19T03:03:55.494052077Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-19T03:03:55.884913661Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-19T03:03:56.249295969Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-19T03:03:56.595010746Z [disk usage checker] Volume usage percentage: current = 49 ; allowed = 70
2026-09-19T03:03:56.817840832Z Gathering data for ns/openshift-multus...
2026-09-19T03:03:58.342152057Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-19T03:03:59.535206488Z Gathering data for ns/openshift-host-network...
2026-09-19T03:03:59.858819345Z Gathering data for ns/openshift-network-diagnostics...
2026-09-19T03:04:00.374273051Z Gathering data for ns/openshift-network-node-identity...
2026-09-19T03:04:00.765991623Z Gathering data for ns/openshift-network-operator...
2026-09-19T03:04:01.121871969Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-19T03:04:01.374576777Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-19T03:04:01.599910554Z [disk usage checker] Volume usage percentage: current = 49 ; allowed = 70
2026-09-19T03:04:01.886229128Z Gathering data for ns/openshift-apiserver-operator...
2026-09-19T03:04:02.269480455Z Gathering data for ns/openshift-apiserver...
2026-09-19T03:04:02.671571736Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-19T03:04:02.864349142Z Collecting kubelet logs for node ip-10-0-133-156.ec2.internal
2026-09-19T03:04:02.866361689Z Collecting kubelet logs for node ip-10-0-137-84.ec2.internal
2026-09-19T03:04:02.867999955Z Collecting kubelet logs for node ip-10-0-130-14.ec2.internal
2026-09-19T03:04:03.024622904Z Gathering data for ns/openshift-controller-manager...
2026-09-19T03:04:03.206894307Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-skwlw namespace
2026-09-19T03:04:03.211977397Z INFO: Node performance data collection complete.
2026-09-19T03:04:03.347441332Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-19T03:04:03.976398696Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-19T03:04:04.306125596Z Gathering data for ns/openshift-service-ca-operator...
2026-09-19T03:04:04.598347776Z Gathering data for ns/openshift-service-ca...
2026-09-19T03:04:04.953392596Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-19T03:04:05.800911740Z Wrote inspect data to must-gather.
2026-09-19T03:04:05.800984505Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T03:04:05.800984505Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-19T03:04:05.923093211Z error: the server doesn't have a resource type "clusters"
2026-09-19T03:04:06.050850114Z Caches written to disk