2026-07-14T07:45:11.419524837Z [disk usage checker] Started
2026-07-14T07:45:11.423348207Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-14T07:45:11.617732672Z Gathering data for ns/openshift-cluster-version...
2026-07-14T07:45:11.739550071Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-14T07:45:11.819250639Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-14T07:45:11.891810984Z Gathering data for ns/default...
2026-07-14T07:45:12.157666746Z Gathering data for ns/openshift...
2026-07-14T07:45:12.555717415Z Gathering data for ns/kube-system...
2026-07-14T07:45:13.222258123Z Gathering data for ns/openshift-etcd...
2026-07-14T07:45:13.514050912Z Wrote inspect data to must-gather.
2026-07-14T07:45:13.514092862Z error: inspection completed with the errors occurred while gathering data:
2026-07-14T07:45:13.514092862Z     namespaces "assisted-installer" not found
2026-07-14T07:45:13.973439708Z Waiting on subprocesses to finish execution.
2026-07-14T07:45:13.981850514Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-14T07:45:13.982243650Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-14T07:45:13.984382912Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-14T07:45:13.988584563Z Executing Istio gather script
2026-07-14T07:45:13.992084482Z INFO: Collecting host service logs for crio
2026-07-14T07:45:13.992431251Z INFO: Collecting host service logs for kubelet
2026-07-14T07:45:13.992756690Z INFO: Collecting host service logs for rpm-ostreed
2026-07-14T07:45:13.993083053Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-14T07:45:13.993437854Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-14T07:45:13.993764819Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-14T07:45:13.994105335Z INFO: Collecting host service logs for NetworkManager
2026-07-14T07:45:13.994499290Z INFO: Collecting host service logs for openvswitch
2026-07-14T07:45:13.994874854Z INFO: Collecting host service logs for ovs-configuration
2026-07-14T07:45:13.995347893Z INFO: Collecting host service logs for ovsdb-server
2026-07-14T07:45:13.995688966Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-14T07:45:13.996108143Z INFO: Waiting for worker host service log collection to complete ...
2026-07-14T07:45:13.999041768Z INFO: Gathering HAProxy config files
2026-07-14T07:45:14.122944836Z INFO: Waiting for node performance related collection to complete ...
2026-07-14T07:45:14.701188371Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-14T07:45:14.903238713Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-646q9
2026-07-14T07:45:14.903294498Z insights-runtime-extractor-dpxcg
2026-07-14T07:45:14.903315506Z insights-runtime-extractor-m4j2j
2026-07-14T07:45:14.903335876Z insights-runtime-extractor-m9d7m
2026-07-14T07:45:14.960675197Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-14T07:45:15.073322326Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-14T07:45:15.193723335Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-14T07:45:15.200394189Z No resources found in openshift-etcd namespace.
2026-07-14T07:45:15.232037833Z Wrote inspect data to must-gather.
2026-07-14T07:45:15.246110935Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-14T07:45:15.246110935Z See 'oc rsync -h' for help and examples
2026-07-14T07:45:15.545837194Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-14T07:45:15.545965303Z INFO: on-disk MachineConfig config collection complete.
2026-07-14T07:45:15.652735166Z INFO: "metallb-operator" not detected. Skipping.
2026-07-14T07:45:15.817867977Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-14T07:45:15.888339011Z ERROR: No running kube-apiserver pods found
2026-07-14T07:45:15.899859112Z Wrote inspect data to must-gather.
2026-07-14T07:45:15.962815941Z error: the server doesn't have a resource type "performanceprofile"
2026-07-14T07:45:15.980996371Z No resources found
2026-07-14T07:45:15.984630117Z No resources found
2026-07-14T07:45:16.011345712Z Wrote inspect data to must-gather.
2026-07-14T07:45:16.176941438Z Wrote inspect data to must-gather.
2026-07-14T07:45:16.204976145Z INFO: Worker host service log collection to complete.
2026-07-14T07:45:16.313492989Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-14T07:45:16.344064218Z Wrote inspect data to must-gather.
2026-07-14T07:45:16.345210514Z INFO: 'previous-logs' folder not found on ip-10-0-131-92.ec2.internal, skipping...
2026-07-14T07:45:16.362796409Z Wrote inspect data to must-gather.
2026-07-14T07:45:16.394763276Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-14T07:45:16.429260381Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-14T07:45:16.432204508Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-14T07:45:16.589766611Z Wrote inspect data to must-gather.
2026-07-14T07:45:16.658405237Z Wrote inspect data to must-gather.
2026-07-14T07:45:16.723594067Z Gathering data for ns/openshift-monitoring...
2026-07-14T07:45:16.733516784Z INFO: 'previous-logs' folder not found on ip-10-0-133-204.ec2.internal, skipping...
2026-07-14T07:45:16.753421261Z tar: Removing leading `/' from member names
2026-07-14T07:45:16.762734449Z INFO: HAProxy config collection complete.
2026-07-14T07:45:16.863929598Z error: the server doesn't have a resource type "machineconfigs"
2026-07-14T07:45:16.889877435Z INFO: Getting rules from prometheus-k8s-0
2026-07-14T07:45:16.931296441Z Wrote inspect data to must-gather.
2026-07-14T07:45:17.046711870Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-14T07:45:17.063728022Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-14T07:45:17.067475711Z INFO: 'previous-logs' folder not found on ip-10-0-142-3.ec2.internal, skipping...
2026-07-14T07:45:17.067514513Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-14T07:45:17.067562556Z INFO: Machine Config Daemon termination log collection complete.
2026-07-14T07:45:17.218236098Z Wrote inspect data to must-gather.
2026-07-14T07:45:17.277362802Z Wrote inspect data to must-gather.
2026-07-14T07:45:17.342963828Z INFO: Getting status/config from prometheus-k8s-0
2026-07-14T07:45:17.446855854Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-14T07:45:17.507448253Z INFO: INTERCONNECT MODE
2026-07-14T07:45:17.507543522Z INFO: Gathering ovn-kubernetes DBs
2026-07-14T07:45:17.607397463Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-14T07:45:17.654392935Z Wrote inspect data to must-gather.
2026-07-14T07:45:17.675729041Z INFO: Gathering OVN_Northbound from ovnkube-node-82s9m...
2026-07-14T07:45:17.675984351Z INFO: Gathering OVN_Northbound from ovnkube-node-c46cz...
2026-07-14T07:45:17.676325533Z INFO: Gathering OVN_Northbound from ovnkube-node-kpq9q...
2026-07-14T07:45:17.676613549Z INFO: Gathering OVN_Southbound from ovnkube-node-82s9m...
2026-07-14T07:45:17.677074571Z INFO: Gathering OVN_Southbound from ovnkube-node-c46cz...
2026-07-14T07:45:17.677390155Z INFO: Gathering OVN_Southbound from ovnkube-node-kpq9q...
2026-07-14T07:45:17.797993701Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-14T07:45:17.931377102Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:17.979158048Z tar: Removing leading `/' from member names
2026-07-14T07:45:17.984945187Z Wrote inspect data to must-gather.
2026-07-14T07:45:17.993978336Z tar: Removing leading `/' from member names
2026-07-14T07:45:18.017243831Z tar: Removing leading `/' from member names
2026-07-14T07:45:18.017930813Z tar: Removing leading `/' from member names
2026-07-14T07:45:18.040456590Z tar: Removing leading `/' from member names
2026-07-14T07:45:18.074644801Z tar: Removing leading `/' from member names
2026-07-14T07:45:18.141350571Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-14T07:45:18.194162888Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:18.196379022Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-14T07:45:18.258158902Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-14T07:45:18.266263660Z Wrote inspect data to must-gather.
2026-07-14T07:45:18.412557075Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-14T07:45:18.423440435Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:18.515646855Z Wrote inspect data to must-gather.
2026-07-14T07:45:18.609248418Z INFO: Waiting for network log collection to complete ...
2026-07-14T07:45:18.609284661Z INFO: Waiting for ovnk database copies to complete ...
2026-07-14T07:45:18.609671768Z INFO: Copying ovnk databases complete.
2026-07-14T07:45:18.611420501Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-14T07:45:18.613653376Z ovnk_database_store/
2026-07-14T07:45:18.613653376Z ovnk_database_store/ovnkube-node-c46cz_nbdb
2026-07-14T07:45:18.625088136Z ovnk_database_store/ovnkube-node-kpq9q_sbdb
2026-07-14T07:45:18.642645636Z ovnk_database_store/ovnkube-node-82s9m_nbdb
2026-07-14T07:45:18.652906226Z ovnk_database_store/ovnkube-node-kpq9q_nbdb
2026-07-14T07:45:18.659225034Z ovnk_database_store/ovnkube-node-82s9m_sbdb
2026-07-14T07:45:18.665753518Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:18.668244284Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-14T07:45:18.669163180Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-14T07:45:18.673439510Z ovnk_database_store/ovnkube-node-c46cz_sbdb
2026-07-14T07:45:18.692199023Z INFO: Network log collection complete.
2026-07-14T07:45:18.756738298Z Wrote inspect data to must-gather.
2026-07-14T07:45:18.869769851Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:18.911485453Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-14T07:45:19.080282536Z Wrote inspect data to must-gather.
2026-07-14T07:45:19.088839945Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:19.092985960Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-14T07:45:19.095853054Z INFO: Getting status from alertmanager-main-0
2026-07-14T07:45:19.122880978Z Gathering data for ns/openshift-network-console...
2026-07-14T07:45:19.346741320Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:19.376082071Z W0714 07:45:19.376030    1570 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-14T07:45:19.390367034Z Wrote inspect data to must-gather.
2026-07-14T07:45:19.563652858Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:19.566284945Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-14T07:45:19.574269875Z Gathering data for ns/openshift-console-operator...
2026-07-14T07:45:19.674415054Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-14T07:45:19.677313786Z INFO: Fallback to identify the container image from release info
2026-07-14T07:45:19.747672098Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:19.948215622Z Wrote inspect data to /must-gather/istio.
2026-07-14T07:45:19.960684089Z Gathering data for ns/openshift-console...
2026-07-14T07:45:20.435288055Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-14T07:45:20.457195745Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-14T07:45:20.603403322Z daemonset.apps/perf-node-gather-daemonset created
2026-07-14T07:45:20.631300816Z Done executing Istio gather script
2026-07-14T07:45:20.727316720Z Waiting for performance profile collector pods to become ready: 1
2026-07-14T07:45:20.933423185Z Gathering data for ns/openshift-dns-operator...
2026-07-14T07:45:21.212581156Z Gathering data for ns/openshift-dns...
2026-07-14T07:45:21.439404686Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-14T07:45:21.764236259Z Gathering data for ns/openshift-image-registry...
2026-07-14T07:45:21.835417190Z Waiting for performance profile collector pods to become ready: 2
2026-07-14T07:45:22.189089489Z Gathering data for ns/openshift-ingress-operator...
2026-07-14T07:45:22.470370465Z Gathering data for ns/openshift-ingress...
2026-07-14T07:45:22.782425192Z Gathering data for ns/openshift-ingress-canary...
2026-07-14T07:45:22.960917479Z Waiting for performance profile collector pods to become ready: 3
2026-07-14T07:45:23.135101938Z Gathering data for ns/openshift-insights...
2026-07-14T07:45:24.067218194Z Waiting for performance profile collector pods to become ready: 4
2026-07-14T07:45:24.917006144Z Gathering data for ns/openshift-config...
2026-07-14T07:45:25.185532581Z Waiting for performance profile collector pods to become ready: 5
2026-07-14T07:45:25.216725341Z Gathering data for ns/openshift-config-managed...
2026-07-14T07:45:25.609437925Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-14T07:45:25.843844304Z Gathering data for ns/openshift-kube-apiserver...
2026-07-14T07:45:26.089515089Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-14T07:45:26.303560729Z Waiting for performance profile collector pods to become ready: 6
2026-07-14T07:45:26.366333640Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-14T07:45:26.445364772Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-14T07:45:26.611927469Z Gathering data for ns/openshift-kube-scheduler...
2026-07-14T07:45:26.840282793Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-14T07:45:27.092202511Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-14T07:45:27.393659790Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-14T07:45:27.418726020Z Waiting for performance profile collector pods to become ready: 7
2026-07-14T07:45:27.725237724Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-14T07:45:28.158432189Z Gathering data for ns/openshift-multus...
2026-07-14T07:45:28.529789193Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-14T07:45:28.753532074Z Collecting performance related data for node ip-10-0-133-204.ec2.internal
2026-07-14T07:45:28.760957198Z Collecting performance related data for node ip-10-0-142-3.ec2.internal
2026-07-14T07:45:28.768461267Z Collecting performance related data for node ip-10-0-131-92.ec2.internal
2026-07-14T07:45:29.617823372Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-14T07:45:30.849774875Z Gathering data for ns/openshift-host-network...
2026-07-14T07:45:31.177889859Z Gathering data for ns/openshift-network-diagnostics...
2026-07-14T07:45:31.450075696Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-14T07:45:31.755314958Z Gathering data for ns/openshift-network-node-identity...
2026-07-14T07:45:32.186059379Z Gathering data for ns/openshift-network-operator...
2026-07-14T07:45:32.566686498Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-14T07:45:32.804806698Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-14T07:45:33.320866878Z Gathering data for ns/openshift-apiserver-operator...
2026-07-14T07:45:33.600551356Z Gathering data for ns/openshift-apiserver...
2026-07-14T07:45:33.995244213Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-14T07:45:34.239481069Z Gathering data for ns/openshift-controller-manager...
2026-07-14T07:45:34.494830015Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-14T07:45:35.133258756Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-14T07:45:35.397302367Z Gathering data for ns/openshift-service-ca-operator...
2026-07-14T07:45:35.704511322Z Gathering data for ns/openshift-service-ca...
2026-07-14T07:45:36.037424886Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-14T07:45:36.454897674Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-14T07:45:36.876409229Z Wrote inspect data to must-gather.
2026-07-14T07:45:36.876456313Z error: inspection completed with the errors occurred while gathering data:
2026-07-14T07:45:36.876456313Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-14T07:45:36.921735764Z Collecting kubelet logs for node ip-10-0-133-204.ec2.internal
2026-07-14T07:45:36.923842191Z Collecting kubelet logs for node ip-10-0-142-3.ec2.internal
2026-07-14T07:45:36.925767187Z Collecting kubelet logs for node ip-10-0-131-92.ec2.internal
2026-07-14T07:45:37.225225114Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-kz4j9 namespace
2026-07-14T07:45:37.230495808Z INFO: Node performance data collection complete.
2026-07-14T07:45:37.358686087Z error: the server doesn't have a resource type "clusters"
2026-07-14T07:45:37.522419229Z Caches written to disk