2026-07-29T15:19:35.433408965Z [disk usage checker] Started
2026-07-29T15:19:35.436859477Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-29T15:19:35.637911770Z Gathering data for ns/openshift-cluster-version...
2026-07-29T15:19:35.772530144Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T15:19:35.866352370Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T15:19:35.952763140Z Gathering data for ns/default...
2026-07-29T15:19:36.253875543Z Gathering data for ns/openshift...
2026-07-29T15:19:36.669644783Z Gathering data for ns/kube-system...
2026-07-29T15:19:37.376094834Z Gathering data for ns/openshift-etcd...
2026-07-29T15:19:37.705078377Z Wrote inspect data to must-gather.
2026-07-29T15:19:37.705111855Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T15:19:37.705111855Z     namespaces "assisted-installer" not found
2026-07-29T15:19:38.169131035Z Waiting on subprocesses to finish execution.
2026-07-29T15:19:38.177442670Z Executing Istio gather script
2026-07-29T15:19:38.180042407Z INFO: Gathering HAProxy config files
2026-07-29T15:19:38.181626369Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-29T15:19:38.189543051Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-29T15:19:38.190594585Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-29T15:19:38.205289627Z INFO: Waiting for node performance related collection to complete ...
2026-07-29T15:19:38.235215779Z INFO: Collecting host service logs for crio
2026-07-29T15:19:38.235545747Z INFO: Collecting host service logs for kubelet
2026-07-29T15:19:38.235906281Z INFO: Collecting host service logs for rpm-ostreed
2026-07-29T15:19:38.236210395Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-29T15:19:38.236511160Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-29T15:19:38.236826837Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-29T15:19:38.237134840Z INFO: Collecting host service logs for NetworkManager
2026-07-29T15:19:38.237416774Z INFO: Collecting host service logs for openvswitch
2026-07-29T15:19:38.237708074Z INFO: Collecting host service logs for ovs-configuration
2026-07-29T15:19:38.238014176Z INFO: Collecting host service logs for ovsdb-server
2026-07-29T15:19:38.238305469Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-29T15:19:38.238657849Z INFO: Waiting for worker host service log collection to complete ...
2026-07-29T15:19:38.622953456Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-bmvmg
2026-07-29T15:19:38.623022437Z insights-runtime-extractor-6pkx4
2026-07-29T15:19:38.623045648Z insights-runtime-extractor-vcx99
2026-07-29T15:19:38.623065671Z insights-runtime-extractor-wpbjl
2026-07-29T15:19:38.734911130Z No resources found in openshift-etcd namespace.
2026-07-29T15:19:38.773354213Z error: the server doesn't have a resource type "performanceprofile"
2026-07-29T15:19:39.044600205Z No resources found
2026-07-29T15:19:39.064415452Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-29T15:19:39.259424329Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-29T15:19:39.259461478Z INFO: on-disk MachineConfig config collection complete.
2026-07-29T15:19:39.335417372Z ERROR: No running kube-apiserver pods found
2026-07-29T15:19:39.384205036Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-29T15:19:39.448679561Z INFO: "metallb-operator" not detected. Skipping.
2026-07-29T15:19:39.557266744Z Wrote inspect data to must-gather.
2026-07-29T15:19:39.760491464Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-29T15:19:39.954356013Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-29T15:19:40.025633754Z INFO: Worker host service log collection to complete.
2026-07-29T15:19:40.034600173Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-29T15:19:40.034600173Z See 'oc rsync -h' for help and examples
2026-07-29T15:19:40.077765975Z Wrote inspect data to must-gather.
2026-07-29T15:19:40.218449156Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T15:19:40.231100311Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-29T15:19:40.238054396Z Wrote inspect data to must-gather.
2026-07-29T15:19:40.418524606Z No resources found
2026-07-29T15:19:40.499711944Z Wrote inspect data to must-gather.
2026-07-29T15:19:40.502217066Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-29T15:19:40.579352025Z Wrote inspect data to must-gather.
2026-07-29T15:19:40.600574928Z Wrote inspect data to must-gather.
2026-07-29T15:19:40.727055433Z INFO: 'previous-logs' folder not found on ip-10-0-133-126.ec2.internal, skipping...
2026-07-29T15:19:40.765547815Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-29T15:19:40.828456774Z error: the server doesn't have a resource type "machineconfigs"
2026-07-29T15:19:40.835335157Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-29T15:19:40.848885467Z Wrote inspect data to must-gather.
2026-07-29T15:19:40.861007932Z tar: Removing leading `/' from member names
2026-07-29T15:19:40.869317866Z INFO: HAProxy config collection complete.
2026-07-29T15:19:41.051582202Z Wrote inspect data to must-gather.
2026-07-29T15:19:41.057899118Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-29T15:19:41.083058395Z INFO: 'previous-logs' folder not found on ip-10-0-140-201.ec2.internal, skipping...
2026-07-29T15:19:41.138400210Z Gathering data for ns/openshift-monitoring...
2026-07-29T15:19:41.335002254Z Wrote inspect data to must-gather.
2026-07-29T15:19:41.360396066Z INFO: Getting rules from prometheus-k8s-0
2026-07-29T15:19:41.365188420Z Wrote inspect data to must-gather.
2026-07-29T15:19:41.435928175Z INFO: 'previous-logs' folder not found on ip-10-0-140-69.ec2.internal, skipping...
2026-07-29T15:19:41.435962113Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-29T15:19:41.435993857Z INFO: Machine Config Daemon termination log collection complete.
2026-07-29T15:19:41.525060989Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-29T15:19:41.664679091Z Wrote inspect data to must-gather.
2026-07-29T15:19:41.772289715Z Wrote inspect data to must-gather.
2026-07-29T15:19:41.808373807Z INFO: Getting status/config from prometheus-k8s-0
2026-07-29T15:19:41.825539927Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-29T15:19:41.997107947Z Wrote inspect data to must-gather.
2026-07-29T15:19:42.018833891Z INFO: INTERCONNECT MODE
2026-07-29T15:19:42.018874298Z INFO: Gathering ovn-kubernetes DBs
2026-07-29T15:19:42.195167112Z INFO: Gathering OVN_Northbound from ovnkube-node-6ddl6...
2026-07-29T15:19:42.195425090Z INFO: Gathering OVN_Northbound from ovnkube-node-lxkcg...
2026-07-29T15:19:42.197547321Z INFO: Gathering OVN_Northbound from ovnkube-node-nc7b4...
2026-07-29T15:19:42.197909613Z INFO: Gathering OVN_Southbound from ovnkube-node-6ddl6...
2026-07-29T15:19:42.198179633Z INFO: Gathering OVN_Southbound from ovnkube-node-lxkcg...
2026-07-29T15:19:42.198457689Z INFO: Gathering OVN_Southbound from ovnkube-node-nc7b4...
2026-07-29T15:19:42.302272139Z Wrote inspect data to must-gather.
2026-07-29T15:19:42.309610012Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-29T15:19:42.571989395Z tar: Removing leading `/' from member names
2026-07-29T15:19:42.585378151Z tar: Removing leading `/' from member names
2026-07-29T15:19:42.591642867Z tar: Removing leading `/' from member names
2026-07-29T15:19:42.595138301Z tar: Removing leading `/' from member names
2026-07-29T15:19:42.600892650Z tar: Removing leading `/' from member names
2026-07-29T15:19:42.627033242Z tar: Removing leading `/' from member names
2026-07-29T15:19:42.642273548Z Wrote inspect data to must-gather.
2026-07-29T15:19:42.706196588Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-29T15:19:42.773030590Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-29T15:19:42.907487715Z Wrote inspect data to must-gather.
2026-07-29T15:19:43.001387003Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-29T15:19:43.007689255Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:43.012863823Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T15:19:43.076056985Z INFO: Waiting for network log collection to complete ...
2026-07-29T15:19:43.076094161Z INFO: Waiting for ovnk database copies to complete ...
2026-07-29T15:19:43.076475659Z INFO: Copying ovnk databases complete.
2026-07-29T15:19:43.078099703Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-29T15:19:43.080335742Z ovnk_database_store/
2026-07-29T15:19:43.080335742Z ovnk_database_store/ovnkube-node-nc7b4_nbdb
2026-07-29T15:19:43.090107986Z ovnk_database_store/ovnkube-node-6ddl6_sbdb
2026-07-29T15:19:43.108383108Z ovnk_database_store/ovnkube-node-6ddl6_nbdb
2026-07-29T15:19:43.115917752Z ovnk_database_store/ovnkube-node-lxkcg_nbdb
2026-07-29T15:19:43.124359825Z ovnk_database_store/ovnkube-node-lxkcg_sbdb
2026-07-29T15:19:43.143213685Z ovnk_database_store/ovnkube-node-nc7b4_sbdb
2026-07-29T15:19:43.159216557Z Wrote inspect data to must-gather.
2026-07-29T15:19:43.165669973Z INFO: Network log collection complete.
2026-07-29T15:19:43.247298511Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:43.250035941Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-29T15:19:43.318167965Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-29T15:19:43.389618978Z Wrote inspect data to must-gather.
2026-07-29T15:19:43.475229912Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:43.592655583Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T15:19:43.695824294Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:43.700938373Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-29T15:19:43.706562690Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-29T15:19:43.708342130Z INFO: Fallback to identify the container image from release info
2026-07-29T15:19:43.758650763Z INFO: Getting status from alertmanager-main-0
2026-07-29T15:19:43.801985644Z Gathering data for ns/openshift-network-console...
2026-07-29T15:19:43.932091185Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:44.124274603Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:44.127148498Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-29T15:19:44.312459524Z Gathering data for ns/openshift-console-operator...
2026-07-29T15:19:44.379413483Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:44.427519814Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-29T15:19:44.557412082Z daemonset.apps/perf-node-gather-daemonset created
2026-07-29T15:19:44.593381463Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:44.597528257Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io
2026-07-29T15:19:44.715993921Z Waiting for performance profile collector pods to become ready: 1
2026-07-29T15:19:44.793804355Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:44.796142732Z Gathering data for ns/openshift-console...
2026-07-29T15:19:44.995766432Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:44.998670142Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io
2026-07-29T15:19:45.198128732Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:45.405031764Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:45.407254962Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io
2026-07-29T15:19:45.412220687Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-29T15:19:45.531121783Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-29T15:19:45.580993667Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:45.791901971Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:45.795934878Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-29T15:19:45.872706690Z Waiting for performance profile collector pods to become ready: 2
2026-07-29T15:19:45.993956068Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:46.023924641Z Gathering data for ns/openshift-dns-operator...
2026-07-29T15:19:46.191243821Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:46.195026463Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-29T15:19:46.351421774Z Gathering data for ns/openshift-dns...
2026-07-29T15:19:46.375170320Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:46.547103442Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:46.549343437Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-29T15:19:46.723272391Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:46.927146805Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:19:47.026570728Z Waiting for performance profile collector pods to become ready: 3
2026-07-29T15:19:47.044053689Z Gathering data for ns/openshift-image-registry...
2026-07-29T15:19:47.634903519Z Done executing Istio gather script
2026-07-29T15:19:47.647739714Z Gathering data for ns/openshift-ingress-operator...
2026-07-29T15:19:48.014098979Z Gathering data for ns/openshift-ingress...
2026-07-29T15:19:48.162998219Z Waiting for performance profile collector pods to become ready: 4
2026-07-29T15:19:48.375122809Z Gathering data for ns/openshift-ingress-canary...
2026-07-29T15:19:48.756184303Z Gathering data for ns/openshift-insights...
2026-07-29T15:19:49.296352832Z Waiting for performance profile collector pods to become ready: 5
2026-07-29T15:19:50.416729338Z Waiting for performance profile collector pods to become ready: 6
2026-07-29T15:19:50.535898793Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-29T15:19:50.996281777Z Gathering data for ns/kserve...
2026-07-29T15:19:51.538478607Z Waiting for performance profile collector pods to become ready: 7
2026-07-29T15:19:52.196307223Z Gathering data for ns/openshift-config...
2026-07-29T15:19:52.464176208Z Gathering data for ns/openshift-config-managed...
2026-07-29T15:19:52.659094785Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-29T15:19:52.918778971Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-29T15:19:52.933326119Z Collecting performance related data for node ip-10-0-140-201.ec2.internal
2026-07-29T15:19:52.942124025Z Collecting performance related data for node ip-10-0-140-69.ec2.internal
2026-07-29T15:19:52.957937584Z Collecting performance related data for node ip-10-0-133-126.ec2.internal
2026-07-29T15:19:53.242134350Z Gathering data for ns/openshift-kube-apiserver...
2026-07-29T15:19:53.613871977Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-29T15:19:53.989698216Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-29T15:19:54.377414814Z Gathering data for ns/openshift-kube-scheduler...
2026-07-29T15:19:54.715903701Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-29T15:19:55.053129784Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-29T15:19:55.436654436Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-29T15:19:55.542981847Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-29T15:19:55.814719166Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-29T15:19:56.395621463Z Gathering data for ns/openshift-multus...
2026-07-29T15:19:57.976270324Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-29T15:19:59.277241243Z Gathering data for ns/openshift-host-network...
2026-07-29T15:19:59.594565342Z Gathering data for ns/openshift-network-diagnostics...
2026-07-29T15:20:00.138269621Z Gathering data for ns/openshift-network-node-identity...
2026-07-29T15:20:00.550816948Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-29T15:20:00.562981601Z Gathering data for ns/openshift-network-operator...
2026-07-29T15:20:00.984019896Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-29T15:20:01.276324000Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-29T15:20:01.355228510Z Collecting kubelet logs for node ip-10-0-140-201.ec2.internal
2026-07-29T15:20:01.357085707Z Collecting kubelet logs for node ip-10-0-140-69.ec2.internal
2026-07-29T15:20:01.358891665Z Collecting kubelet logs for node ip-10-0-133-126.ec2.internal
2026-07-29T15:20:01.717367372Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-lgfhp namespace
2026-07-29T15:20:01.723881613Z INFO: Node performance data collection complete.
2026-07-29T15:20:01.863419396Z Gathering data for ns/openshift-apiserver-operator...
2026-07-29T15:20:02.220537001Z Gathering data for ns/openshift-apiserver...
2026-07-29T15:20:02.620173367Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-29T15:20:02.887347730Z Gathering data for ns/openshift-controller-manager...
2026-07-29T15:20:03.172399020Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-29T15:20:03.818742721Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-29T15:20:04.150855673Z Gathering data for ns/openshift-service-ca-operator...
2026-07-29T15:20:04.458193444Z Gathering data for ns/openshift-service-ca...
2026-07-29T15:20:04.820900812Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-29T15:20:05.555699625Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-29T15:20:05.711772964Z Wrote inspect data to must-gather.
2026-07-29T15:20:05.711853016Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T15:20:05.711853016Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-29T15:20:05.840975182Z error: the server doesn't have a resource type "clusters"
2026-07-29T15:20:06.004831147Z Caches written to disk