2026-09-17T04:02:27.942491274Z [disk usage checker] Started
2026-09-17T04:02:27.946080964Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T04:02:28.152338637Z Gathering data for ns/openshift-cluster-version...
2026-09-17T04:02:28.302066742Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T04:02:28.390407847Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T04:02:28.485861965Z Gathering data for ns/default...
2026-09-17T04:02:28.801167920Z Gathering data for ns/openshift...
2026-09-17T04:02:29.230762755Z Gathering data for ns/kube-system...
2026-09-17T04:02:29.921248845Z Gathering data for ns/openshift-etcd...
2026-09-17T04:02:30.222839716Z Wrote inspect data to must-gather.
2026-09-17T04:02:30.222870717Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T04:02:30.222870717Z     namespaces "assisted-installer" not found
2026-09-17T04:02:30.484853314Z Waiting on subprocesses to finish execution.
2026-09-17T04:02:30.489954232Z INFO: Gathering HAProxy config files
2026-09-17T04:02:30.490385040Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T04:02:30.492636551Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T04:02:30.495731647Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T04:02:30.495862103Z Executing Istio gather script
2026-09-17T04:02:30.505843079Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T04:02:30.524816657Z INFO: Collecting host service logs for crio
2026-09-17T04:02:30.525240107Z INFO: Collecting host service logs for kubelet
2026-09-17T04:02:30.525627536Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T04:02:30.525961937Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T04:02:30.526311419Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T04:02:30.526641096Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T04:02:30.526959844Z INFO: Collecting host service logs for NetworkManager
2026-09-17T04:02:30.527280057Z INFO: Collecting host service logs for openvswitch
2026-09-17T04:02:30.527648346Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T04:02:30.527991233Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T04:02:30.528317798Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T04:02:30.528694299Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T04:02:31.117988212Z INFO: Collecting Insights Archives from insights-operator-5844b856db-jfw9s
2026-09-17T04:02:31.118048302Z insights-runtime-extractor-cgfcs
2026-09-17T04:02:31.119598346Z insights-runtime-extractor-gqtch
2026-09-17T04:02:31.119634145Z insights-runtime-extractor-qprvm
2026-09-17T04:02:31.178722367Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T04:02:31.218743200Z No resources found
2026-09-17T04:02:31.261485919Z Wrote inspect data to must-gather.
2026-09-17T04:02:31.477439656Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T04:02:31.508604938Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T04:02:31.508686070Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T04:02:31.570975159Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T04:02:31.934138640Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T04:02:31.934138640Z See 'oc rsync -h' for help and examples
2026-09-17T04:02:31.966256689Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T04:02:32.006766326Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T04:02:32.021031209Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T04:02:32.047400648Z No resources found in openshift-etcd namespace.
2026-09-17T04:02:32.056075598Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T04:02:32.069862668Z INFO: Worker host service log collection to complete.
2026-09-17T04:02:32.301023577Z Gathering data for ns/openshift-monitoring...
2026-09-17T04:02:32.305108733Z Wrote inspect data to must-gather.
2026-09-17T04:02:32.305761873Z INFO: 'previous-logs' folder not found on ip-10-0-135-144.ec2.internal, skipping...
2026-09-17T04:02:32.644456524Z tar: Removing leading `/' from member names
2026-09-17T04:02:32.647984394Z INFO: HAProxy config collection complete.
2026-09-17T04:02:32.695919107Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T04:02:32.736817142Z ERROR: No running kube-apiserver pods found
2026-09-17T04:02:32.738299054Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T04:02:32.767997812Z Wrote inspect data to must-gather.
2026-09-17T04:02:32.882253791Z Wrote inspect data to must-gather.
2026-09-17T04:02:32.934844632Z No resources found
2026-09-17T04:02:32.949934990Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T04:02:33.013521310Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T04:02:33.165619542Z INFO: 'previous-logs' folder not found on ip-10-0-140-252.ec2.internal, skipping...
2026-09-17T04:02:33.180742349Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T04:02:33.193946677Z Wrote inspect data to must-gather.
2026-09-17T04:02:33.212318938Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T04:02:33.229381196Z Wrote inspect data to must-gather.
2026-09-17T04:02:33.230879788Z Wrote inspect data to must-gather.
2026-09-17T04:02:33.356226381Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T04:02:33.437975201Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T04:02:33.448958616Z INFO: 'previous-logs' folder not found on ip-10-0-142-108.ec2.internal, skipping...
2026-09-17T04:02:33.448990906Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T04:02:33.449019163Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T04:02:33.487156320Z Wrote inspect data to must-gather.
2026-09-17T04:02:33.584480395Z Wrote inspect data to must-gather.
2026-09-17T04:02:33.718220444Z Wrote inspect data to must-gather.
2026-09-17T04:02:33.747601084Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T04:02:33.811964127Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T04:02:33.991408643Z Wrote inspect data to must-gather.
2026-09-17T04:02:34.005192657Z Wrote inspect data to must-gather.
2026-09-17T04:02:34.053125364Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T04:02:34.224278469Z Wrote inspect data to must-gather.
2026-09-17T04:02:34.243910525Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T04:02:34.277412606Z INFO: INTERCONNECT MODE
2026-09-17T04:02:34.277450510Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T04:02:34.450854697Z Wrote inspect data to must-gather.
2026-09-17T04:02:34.454973339Z INFO: Gathering OVN_Northbound from ovnkube-node-85sxn...
2026-09-17T04:02:34.455229510Z INFO: Gathering OVN_Northbound from ovnkube-node-8rtwb...
2026-09-17T04:02:34.455618121Z INFO: Gathering OVN_Northbound from ovnkube-node-x95fr...
2026-09-17T04:02:34.455988182Z INFO: Gathering OVN_Southbound from ovnkube-node-85sxn...
2026-09-17T04:02:34.456294071Z INFO: Gathering OVN_Southbound from ovnkube-node-8rtwb...
2026-09-17T04:02:34.456569764Z INFO: Gathering OVN_Southbound from ovnkube-node-x95fr...
2026-09-17T04:02:34.490196230Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T04:02:34.497847215Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T04:02:34.713357787Z tar: Removing leading `/' from member names
2026-09-17T04:02:34.734295888Z tar: Removing leading `/' from member names
2026-09-17T04:02:34.753479282Z tar: Removing leading `/' from member names
2026-09-17T04:02:34.790035162Z tar: Removing leading `/' from member names
2026-09-17T04:02:34.793559983Z tar: Removing leading `/' from member names
2026-09-17T04:02:34.794073609Z W0917 04:02:34.794029    1322 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-17T04:02:34.801614643Z Wrote inspect data to must-gather.
2026-09-17T04:02:34.825865981Z tar: Removing leading `/' from member names
2026-09-17T04:02:34.868081136Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T04:02:34.883489900Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:34.902110212Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T04:02:35.115668210Z Gathering data for ns/openshift-network-console...
2026-09-17T04:02:35.149809434Z Wrote inspect data to must-gather.
2026-09-17T04:02:35.152144801Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:35.155323109Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T04:02:35.180948325Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T04:02:35.328338903Z INFO: Waiting for network log collection to complete ...
2026-09-17T04:02:35.328368169Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T04:02:35.328740973Z INFO: Copying ovnk databases complete.
2026-09-17T04:02:35.330455252Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-09-17T04:02:35.334911902Z ovnk_database_store/
2026-09-17T04:02:35.334911902Z ovnk_database_store/ovnkube-node-x95fr_nbdb
2026-09-17T04:02:35.341879861Z ovnk_database_store/ovnkube-node-85sxn_nbdb
2026-09-17T04:02:35.349514499Z ovnk_database_store/ovnkube-node-x95fr_sbdb
2026-09-17T04:02:35.367428693Z ovnk_database_store/ovnkube-node-8rtwb_sbdb
2026-09-17T04:02:35.386202767Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:35.386588195Z ovnk_database_store/ovnkube-node-8rtwb_nbdb
2026-09-17T04:02:35.394066622Z ovnk_database_store/ovnkube-node-85sxn_sbdb
2026-09-17T04:02:35.415625150Z INFO: Network log collection complete.
2026-09-17T04:02:35.495671421Z Wrote inspect data to must-gather.
2026-09-17T04:02:35.601053960Z INFO: Getting status from alertmanager-main-0
2026-09-17T04:02:35.620314020Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:35.624989017Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T04:02:35.650720386Z Gathering data for ns/openshift-console-operator...
2026-09-17T04:02:35.816337013Z Wrote inspect data to must-gather.
2026-09-17T04:02:35.845097640Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:36.045354126Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:36.048923255Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T04:02:36.085088085Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T04:02:36.087239330Z INFO: Fallback to identify the container image from release info
2026-09-17T04:02:36.107904276Z Gathering data for ns/openshift-console...
2026-09-17T04:02:36.272974471Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:36.454469527Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:36.456563282Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T04:02:36.631420331Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:36.664121513Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T04:02:36.669545912Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-17T04:02:36.802108157Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T04:02:36.832343828Z Wrote inspect data to /must-gather/istio.
2026-09-17T04:02:36.935413753Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T04:02:37.234774940Z Gathering data for ns/openshift-dns-operator...
2026-09-17T04:02:37.526875454Z Done executing Istio gather script
2026-09-17T04:02:37.573640725Z Gathering data for ns/openshift-dns...
2026-09-17T04:02:38.030297335Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T04:02:38.057516231Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T04:02:38.179221253Z Gathering data for ns/openshift-image-registry...
2026-09-17T04:02:38.669527469Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T04:02:39.019337199Z Gathering data for ns/openshift-ingress...
2026-09-17T04:02:39.170141407Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T04:02:39.353494230Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T04:02:39.720978816Z Gathering data for ns/openshift-insights...
2026-09-17T04:02:40.282659351Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T04:02:41.393589457Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T04:02:41.555019409Z Gathering data for ns/openshift-jobset-operator...
2026-09-17T04:02:42.333381207Z Gathering data for ns/openshift-config...
2026-09-17T04:02:42.505675806Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T04:02:42.619871129Z Gathering data for ns/openshift-config-managed...
2026-09-17T04:02:43.026665692Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T04:02:43.035284488Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T04:02:43.289873589Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T04:02:43.585122270Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T04:02:43.635917472Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T04:02:43.863640700Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T04:02:44.143335727Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T04:02:44.376557850Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T04:02:44.671616819Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T04:02:44.743796033Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T04:02:45.014810602Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T04:02:45.039827102Z Collecting performance related data for node ip-10-0-135-144.ec2.internal
2026-09-17T04:02:45.046831146Z Collecting performance related data for node ip-10-0-140-252.ec2.internal
2026-09-17T04:02:45.056615977Z Collecting performance related data for node ip-10-0-142-108.ec2.internal
2026-09-17T04:02:45.387431387Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T04:02:46.029380169Z Gathering data for ns/openshift-multus...
2026-09-17T04:02:47.580387567Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T04:02:48.040183500Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T04:02:48.832088698Z Gathering data for ns/openshift-host-network...
2026-09-17T04:02:49.137498827Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T04:02:49.676557033Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T04:02:50.151013936Z Gathering data for ns/openshift-network-operator...
2026-09-17T04:02:50.525743759Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T04:02:50.783004027Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T04:02:51.252154879Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T04:02:51.594418454Z Gathering data for ns/openshift-apiserver...
2026-09-17T04:02:52.002520318Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T04:02:52.262091846Z Gathering data for ns/openshift-controller-manager...
2026-09-17T04:02:52.576679118Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T04:02:53.044658452Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-09-17T04:02:53.079959564Z Collecting kubelet logs for node ip-10-0-135-144.ec2.internal
2026-09-17T04:02:53.081667729Z Collecting kubelet logs for node ip-10-0-140-252.ec2.internal
2026-09-17T04:02:53.083361962Z Collecting kubelet logs for node ip-10-0-142-108.ec2.internal
2026-09-17T04:02:53.248046365Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T04:02:53.426645693Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-d4zbp namespace
2026-09-17T04:02:53.431508629Z INFO: Node performance data collection complete.
2026-09-17T04:02:53.552206516Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T04:02:53.869230534Z Gathering data for ns/openshift-service-ca...
2026-09-17T04:02:54.230358181Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T04:02:55.104084171Z Wrote inspect data to must-gather.
2026-09-17T04:02:55.104129929Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T04:02:55.104129929Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T04:02:55.226983717Z error: the server doesn't have a resource type "clusters"
2026-09-17T04:02:55.346242475Z Caches written to disk