2026-09-17T18:28:13.147657921Z [disk usage checker] Started
2026-09-17T18:28:13.151236037Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T18:28:13.333902834Z Gathering data for ns/openshift-cluster-version...
2026-09-17T18:28:13.452750908Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:28:13.542709827Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:28:13.624180384Z Gathering data for ns/default...
2026-09-17T18:28:13.925937708Z Gathering data for ns/openshift...
2026-09-17T18:28:14.317571648Z Gathering data for ns/kube-system...
2026-09-17T18:28:15.008388753Z Gathering data for ns/openshift-etcd...
2026-09-17T18:28:15.293741638Z Wrote inspect data to must-gather.
2026-09-17T18:28:15.293770456Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:28:15.293770456Z     namespaces "assisted-installer" not found
2026-09-17T18:28:15.629004469Z Waiting on subprocesses to finish execution.
2026-09-17T18:28:15.637406054Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T18:28:15.637742795Z Executing Istio gather script
2026-09-17T18:28:15.641777734Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T18:28:15.642347124Z INFO: Gathering HAProxy config files
2026-09-17T18:28:15.645975434Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T18:28:15.663090005Z INFO: Collecting host service logs for crio
2026-09-17T18:28:15.663419783Z INFO: Collecting host service logs for kubelet
2026-09-17T18:28:15.663788720Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T18:28:15.664141493Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T18:28:15.664516556Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T18:28:15.664858197Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T18:28:15.665205349Z INFO: Collecting host service logs for NetworkManager
2026-09-17T18:28:15.665547900Z INFO: Collecting host service logs for openvswitch
2026-09-17T18:28:15.665892195Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T18:28:15.666234941Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T18:28:15.666633136Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T18:28:15.667044279Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T18:28:15.730577224Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T18:28:16.380002230Z No resources found in openshift-etcd namespace.
2026-09-17T18:28:16.511561211Z No resources found
2026-09-17T18:28:16.740345741Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T18:28:16.793648468Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-lvcl5
2026-09-17T18:28:16.793694939Z insights-runtime-extractor-jhgtx
2026-09-17T18:28:16.793715299Z insights-runtime-extractor-nsqg7
2026-09-17T18:28:16.793734957Z insights-runtime-extractor-vm5k7
2026-09-17T18:28:16.904664280Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T18:28:16.904739755Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T18:28:17.187720124Z Wrote inspect data to must-gather.
2026-09-17T18:28:17.217174051Z Wrote inspect data to must-gather.
2026-09-17T18:28:17.259410932Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T18:28:17.298840062Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T18:28:17.361433058Z INFO: Worker host service log collection to complete.
2026-09-17T18:28:17.362282399Z ERROR: No running kube-apiserver pods found
2026-09-17T18:28:17.385936347Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T18:28:17.449703076Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T18:28:17.522351794Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T18:28:17.552187586Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T18:28:17.566249559Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T18:28:17.628217425Z No resources found
2026-09-17T18:28:17.629516571Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T18:28:17.629516571Z See 'oc rsync -h' for help and examples
2026-09-17T18:28:17.646500347Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:28:17.732694068Z Wrote inspect data to must-gather.
2026-09-17T18:28:17.779130857Z Wrote inspect data to must-gather.
2026-09-17T18:28:17.861947313Z Wrote inspect data to must-gather.
2026-09-17T18:28:18.037282851Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T18:28:18.037714879Z tar: Removing leading `/' from member names
2026-09-17T18:28:18.046247259Z INFO: HAProxy config collection complete.
2026-09-17T18:28:18.064560453Z Wrote inspect data to must-gather.
2026-09-17T18:28:18.095439714Z Wrote inspect data to must-gather.
2026-09-17T18:28:18.095979272Z INFO: 'previous-logs' folder not found on ip-10-0-131-253.ec2.internal, skipping...
2026-09-17T18:28:18.161019710Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T18:28:18.272823361Z Gathering data for ns/openshift-monitoring...
2026-09-17T18:28:18.290669399Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T18:28:18.297705080Z Wrote inspect data to must-gather.
2026-09-17T18:28:18.383207801Z INFO: 'previous-logs' folder not found on ip-10-0-133-184.ec2.internal, skipping...
2026-09-17T18:28:18.478539847Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T18:28:18.513705408Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T18:28:18.576605041Z Wrote inspect data to must-gather.
2026-09-17T18:28:18.717409249Z Wrote inspect data to must-gather.
2026-09-17T18:28:18.738737394Z INFO: 'previous-logs' folder not found on ip-10-0-137-98.ec2.internal, skipping...
2026-09-17T18:28:18.738768392Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T18:28:18.738792197Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T18:28:18.754992236Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T18:28:18.848772731Z Wrote inspect data to must-gather.
2026-09-17T18:28:18.868583404Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T18:28:18.934521033Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T18:28:19.099954480Z Wrote inspect data to must-gather.
2026-09-17T18:28:19.126179148Z INFO: INTERCONNECT MODE
2026-09-17T18:28:19.126203689Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T18:28:19.276200872Z INFO: Gathering OVN_Northbound from ovnkube-node-46v77...
2026-09-17T18:28:19.276441131Z INFO: Gathering OVN_Northbound from ovnkube-node-lsjv9...
2026-09-17T18:28:19.276734694Z INFO: Gathering OVN_Northbound from ovnkube-node-t2w6b...
2026-09-17T18:28:19.276987664Z INFO: Gathering OVN_Southbound from ovnkube-node-46v77...
2026-09-17T18:28:19.277298112Z INFO: Gathering OVN_Southbound from ovnkube-node-lsjv9...
2026-09-17T18:28:19.277561042Z INFO: Gathering OVN_Southbound from ovnkube-node-t2w6b...
2026-09-17T18:28:19.311593753Z Wrote inspect data to must-gather.
2026-09-17T18:28:19.422885695Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T18:28:19.541250687Z tar: Removing leading `/' from member names
2026-09-17T18:28:19.543433973Z tar: Removing leading `/' from member names
2026-09-17T18:28:19.560838416Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T18:28:19.563470877Z tar: Removing leading `/' from member names
2026-09-17T18:28:19.589077859Z tar: Removing leading `/' from member names
2026-09-17T18:28:19.619035897Z tar: Removing leading `/' from member names
2026-09-17T18:28:19.636241607Z tar: Removing leading `/' from member names
2026-09-17T18:28:19.637554743Z Wrote inspect data to must-gather.
2026-09-17T18:28:19.727613390Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T18:28:19.790957321Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:19.807030100Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T18:28:19.890414650Z Wrote inspect data to must-gather.
2026-09-17T18:28:19.982908553Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T18:28:20.050212203Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:20.052967364Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T18:28:20.128667750Z Wrote inspect data to must-gather.
2026-09-17T18:28:20.137098777Z INFO: Waiting for network log collection to complete ...
2026-09-17T18:28:20.137176677Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T18:28:20.137632101Z INFO: Copying ovnk databases complete.
2026-09-17T18:28:20.139613640Z 3.0M	must-gather/network_logs/ovnk_database_store
2026-09-17T18:28:20.142124589Z ovnk_database_store/
2026-09-17T18:28:20.142124589Z ovnk_database_store/ovnkube-node-46v77_sbdb
2026-09-17T18:28:20.160043236Z ovnk_database_store/ovnkube-node-lsjv9_nbdb
2026-09-17T18:28:20.168611928Z ovnk_database_store/ovnkube-node-t2w6b_nbdb
2026-09-17T18:28:20.177186760Z ovnk_database_store/ovnkube-node-lsjv9_sbdb
2026-09-17T18:28:20.195911806Z ovnk_database_store/ovnkube-node-46v77_nbdb
2026-09-17T18:28:20.202684184Z ovnk_database_store/ovnkube-node-t2w6b_sbdb
2026-09-17T18:28:20.221798899Z INFO: Network log collection complete.
2026-09-17T18:28:20.248507360Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T18:28:20.281427157Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:20.333384131Z W0917 18:28:20.333328    1505 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-17T18:28:20.338690644Z Wrote inspect data to must-gather.
2026-09-17T18:28:20.454901237Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T18:28:20.482765461Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:20.485437985Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T18:28:20.625559601Z INFO: Getting status from alertmanager-main-0
2026-09-17T18:28:20.631824816Z Wrote inspect data to must-gather.
2026-09-17T18:28:20.656475245Z Gathering data for ns/openshift-network-console...
2026-09-17T18:28:20.692073171Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:20.898337038Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T18:28:20.900612053Z INFO: Fallback to identify the container image from release info
2026-09-17T18:28:20.914212531Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:20.917728772Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T18:28:21.127559408Z Gathering data for ns/openshift-console-operator...
2026-09-17T18:28:21.145725519Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:21.320167800Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:21.322642611Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T18:28:21.486541549Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:21.502745159Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-17T18:28:21.508123196Z Gathering data for ns/openshift-console...
2026-09-17T18:28:21.633489354Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T18:28:21.696782294Z Wrote inspect data to /must-gather/istio.
2026-09-17T18:28:21.751496782Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T18:28:22.057009742Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T18:28:22.377972509Z Done executing Istio gather script
2026-09-17T18:28:22.542913466Z Gathering data for ns/openshift-dns-operator...
2026-09-17T18:28:22.804878792Z Gathering data for ns/openshift-dns...
2026-09-17T18:28:22.873868002Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T18:28:23.166160278Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T18:28:23.351535240Z Gathering data for ns/openshift-image-registry...
2026-09-17T18:28:23.770691376Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T18:28:23.975221876Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T18:28:24.120125071Z Gathering data for ns/openshift-ingress...
2026-09-17T18:28:24.430465413Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T18:28:24.782518475Z Gathering data for ns/openshift-insights...
2026-09-17T18:28:25.094121857Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T18:28:26.195824047Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T18:28:26.527430106Z Gathering data for ns/openshift-jobset-operator...
2026-09-17T18:28:27.299466043Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T18:28:27.309110638Z Gathering data for ns/openshift-config...
2026-09-17T18:28:27.581129005Z Gathering data for ns/openshift-config-managed...
2026-09-17T18:28:27.992852595Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T18:28:28.171717183Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T18:28:28.239656006Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T18:28:28.404101946Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T18:28:28.492909400Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T18:28:28.759654215Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T18:28:29.021946806Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T18:28:29.258635105Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T18:28:29.529200545Z Waiting for performance profile collector pods to become ready: 8
2026-09-17T18:28:29.540249136Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T18:28:29.845670615Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T18:28:30.134395644Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T18:28:30.593953507Z Gathering data for ns/openshift-multus...
2026-09-17T18:28:30.639066589Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T18:28:30.854115673Z Collecting performance related data for node ip-10-0-131-253.ec2.internal
2026-09-17T18:28:30.860692991Z Collecting performance related data for node ip-10-0-133-184.ec2.internal
2026-09-17T18:28:30.867614424Z Collecting performance related data for node ip-10-0-137-98.ec2.internal
2026-09-17T18:28:32.039865802Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T18:28:33.179155280Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T18:28:33.289145832Z Gathering data for ns/openshift-host-network...
2026-09-17T18:28:33.691592796Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T18:28:34.201637588Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T18:28:34.604787922Z Gathering data for ns/openshift-network-operator...
2026-09-17T18:28:34.988593804Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T18:28:35.272958783Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T18:28:35.747225691Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T18:28:36.046018296Z Gathering data for ns/openshift-apiserver...
2026-09-17T18:28:36.479111220Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T18:28:36.788436836Z Gathering data for ns/openshift-controller-manager...
2026-09-17T18:28:37.048441456Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T18:28:37.733631848Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T18:28:38.099984583Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T18:28:38.185439638Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T18:28:38.435758787Z Gathering data for ns/openshift-service-ca...
2026-09-17T18:28:38.658907253Z Collecting kubelet logs for node ip-10-0-131-253.ec2.internal
2026-09-17T18:28:38.660632777Z Collecting kubelet logs for node ip-10-0-133-184.ec2.internal
2026-09-17T18:28:38.662774343Z Collecting kubelet logs for node ip-10-0-137-98.ec2.internal
2026-09-17T18:28:38.843576585Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T18:28:39.001003532Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-bjkhp namespace
2026-09-17T18:28:39.005598017Z INFO: Node performance data collection complete.
2026-09-17T18:28:39.717784309Z Wrote inspect data to must-gather.
2026-09-17T18:28:39.717824400Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T18:28:39.717824400Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T18:28:39.837216890Z error: the server doesn't have a resource type "clusters"
2026-09-17T18:28:39.962597352Z Caches written to disk