2026-07-20T19:24:15.371866704Z [disk usage checker] Started
2026-07-20T19:24:15.375644981Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-20T19:24:15.566813743Z Gathering data for ns/openshift-cluster-version...
2026-07-20T19:24:15.683474357Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T19:24:15.760772599Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:24:15.838767311Z Gathering data for ns/default...
2026-07-20T19:24:16.146180865Z Gathering data for ns/openshift...
2026-07-20T19:24:16.522972864Z Gathering data for ns/kube-system...
2026-07-20T19:24:17.204709164Z Gathering data for ns/openshift-etcd...
2026-07-20T19:24:17.504035660Z Wrote inspect data to must-gather.
2026-07-20T19:24:17.504082860Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T19:24:17.504082860Z     namespaces "assisted-installer" not found
2026-07-20T19:24:17.790438818Z Waiting on subprocesses to finish execution.
2026-07-20T19:24:17.797060212Z INFO: Gathering HAProxy config files
2026-07-20T19:24:17.801503717Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-20T19:24:17.802341301Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-20T19:24:17.804477203Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-20T19:24:17.808542873Z INFO: Collecting host service logs for crio
2026-07-20T19:24:17.808910845Z INFO: Collecting host service logs for kubelet
2026-07-20T19:24:17.809294062Z INFO: Collecting host service logs for rpm-ostreed
2026-07-20T19:24:17.809569258Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-20T19:24:17.809852580Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-20T19:24:17.810136424Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-20T19:24:17.810435977Z INFO: Collecting host service logs for NetworkManager
2026-07-20T19:24:17.810714820Z INFO: Collecting host service logs for openvswitch
2026-07-20T19:24:17.810993357Z INFO: Collecting host service logs for ovs-configuration
2026-07-20T19:24:17.811365179Z INFO: Collecting host service logs for ovsdb-server
2026-07-20T19:24:17.811743197Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-20T19:24:17.811932401Z Executing Istio gather script
2026-07-20T19:24:17.812168003Z INFO: Waiting for worker host service log collection to complete ...
2026-07-20T19:24:17.950336729Z INFO: Waiting for node performance related collection to complete ...
2026-07-20T19:24:18.069889238Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-20T19:24:18.413716979Z No resources found in openshift-etcd namespace.
2026-07-20T19:24:18.501557907Z Wrote inspect data to must-gather.
2026-07-20T19:24:18.612676144Z No resources found
2026-07-20T19:24:18.620655749Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-20T19:24:18.800843389Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-6sbnn
2026-07-20T19:24:18.800886277Z insights-runtime-extractor-77zh7
2026-07-20T19:24:18.800904015Z insights-runtime-extractor-fgxnt
2026-07-20T19:24:18.800920967Z insights-runtime-extractor-s6lmn
2026-07-20T19:24:18.809864316Z INFO: "metallb-operator" not detected. Skipping.
2026-07-20T19:24:19.016809435Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-20T19:24:19.067776642Z error: the server doesn't have a resource type "performanceprofile"
2026-07-20T19:24:19.121505525Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-20T19:24:19.644310538Z ERROR: No running kube-apiserver pods found
2026-07-20T19:24:19.743891120Z Wrote inspect data to must-gather.
2026-07-20T19:24:19.795850164Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-20T19:24:19.795850164Z See 'oc rsync -h' for help and examples
2026-07-20T19:24:19.806836023Z Wrote inspect data to must-gather.
2026-07-20T19:24:19.829523208Z INFO: Worker host service log collection to complete.
2026-07-20T19:24:19.831982927Z Wrote inspect data to must-gather.
2026-07-20T19:24:19.834091355Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-20T19:24:19.838256948Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-20T19:24:19.838372340Z INFO: on-disk MachineConfig config collection complete.
2026-07-20T19:24:19.872149843Z Wrote inspect data to must-gather.
2026-07-20T19:24:19.923917536Z Wrote inspect data to must-gather.
2026-07-20T19:24:20.035648550Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-20T19:24:20.109232466Z No resources found
2026-07-20T19:24:20.113285908Z Wrote inspect data to must-gather.
2026-07-20T19:24:20.232920971Z INFO: 'previous-logs' folder not found on ip-10-0-130-194.ec2.internal, skipping...
2026-07-20T19:24:20.254309351Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:24:20.260820932Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-20T19:24:20.270197800Z error: the server doesn't have a resource type "machineconfigs"
2026-07-20T19:24:20.365224041Z tar: Removing leading `/' from member names
2026-07-20T19:24:20.368019447Z Gathering data for ns/openshift-monitoring...
2026-07-20T19:24:20.371323357Z INFO: HAProxy config collection complete.
2026-07-20T19:24:20.393571129Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-20T19:24:20.477218376Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-20T19:24:20.522558685Z Wrote inspect data to must-gather.
2026-07-20T19:24:20.582200176Z INFO: 'previous-logs' folder not found on ip-10-0-133-155.ec2.internal, skipping...
2026-07-20T19:24:20.747308497Z Wrote inspect data to must-gather.
2026-07-20T19:24:20.769823067Z INFO: Getting rules from prometheus-k8s-0
2026-07-20T19:24:20.819621520Z Wrote inspect data to must-gather.
2026-07-20T19:24:20.839731561Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-20T19:24:20.870565201Z INFO: 'previous-logs' folder not found on ip-10-0-135-182.ec2.internal, skipping...
2026-07-20T19:24:20.870601007Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-20T19:24:20.870633659Z INFO: Machine Config Daemon termination log collection complete.
2026-07-20T19:24:20.919432430Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-20T19:24:21.069616585Z Wrote inspect data to must-gather.
2026-07-20T19:24:21.121816634Z Wrote inspect data to must-gather.
2026-07-20T19:24:21.190984900Z INFO: Getting status/config from prometheus-k8s-0
2026-07-20T19:24:21.330377859Z INFO: INTERCONNECT MODE
2026-07-20T19:24:21.330405147Z INFO: Gathering ovn-kubernetes DBs
2026-07-20T19:24:21.350647543Z Wrote inspect data to must-gather.
2026-07-20T19:24:21.490261711Z INFO: Gathering OVN_Northbound from ovnkube-node-d4w7w...
2026-07-20T19:24:21.490531837Z INFO: Gathering OVN_Northbound from ovnkube-node-j4c9c...
2026-07-20T19:24:21.490760969Z INFO: Gathering OVN_Northbound from ovnkube-node-rh6f4...
2026-07-20T19:24:21.491062659Z INFO: Gathering OVN_Southbound from ovnkube-node-d4w7w...
2026-07-20T19:24:21.491324656Z INFO: Gathering OVN_Southbound from ovnkube-node-j4c9c...
2026-07-20T19:24:21.491615618Z INFO: Gathering OVN_Southbound from ovnkube-node-rh6f4...
2026-07-20T19:24:21.564465139Z Wrote inspect data to must-gather.
2026-07-20T19:24:21.721995951Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-20T19:24:21.800956658Z tar: Removing leading `/' from member names
2026-07-20T19:24:21.811077768Z tar: Removing leading `/' from member names
2026-07-20T19:24:21.829908360Z tar: Removing leading `/' from member names
2026-07-20T19:24:21.832558642Z tar: Removing leading `/' from member names
2026-07-20T19:24:21.865636078Z tar: Removing leading `/' from member names
2026-07-20T19:24:21.869614793Z tar: Removing leading `/' from member names
2026-07-20T19:24:21.920521880Z Wrote inspect data to must-gather.
2026-07-20T19:24:21.920934782Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-20T19:24:22.014693240Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-20T19:24:22.140547907Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:22.143086267Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T19:24:22.180375290Z Wrote inspect data to must-gather.
2026-07-20T19:24:22.324150193Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-20T19:24:22.373663289Z INFO: Waiting for network log collection to complete ...
2026-07-20T19:24:22.373780309Z INFO: Waiting for ovnk database copies to complete ...
2026-07-20T19:24:22.374379273Z INFO: Copying ovnk databases complete.
2026-07-20T19:24:22.376435445Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-20T19:24:22.379780213Z ovnk_database_store/
2026-07-20T19:24:22.379824434Z ovnk_database_store/ovnkube-node-j4c9c_sbdb
2026-07-20T19:24:22.387571618Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:22.390480267Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-20T19:24:22.402136023Z ovnk_database_store/ovnkube-node-d4w7w_nbdb
2026-07-20T19:24:22.409247900Z ovnk_database_store/ovnkube-node-d4w7w_sbdb
2026-07-20T19:24:22.425488987Z ovnk_database_store/ovnkube-node-j4c9c_nbdb
2026-07-20T19:24:22.431093536Z ovnk_database_store/ovnkube-node-rh6f4_nbdb
2026-07-20T19:24:22.434573863Z Wrote inspect data to must-gather.
2026-07-20T19:24:22.436564265Z ovnk_database_store/ovnkube-node-rh6f4_sbdb
2026-07-20T19:24:22.457679332Z INFO: Network log collection complete.
2026-07-20T19:24:22.579206870Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-20T19:24:22.589448961Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:22.634359227Z W0720 19:24:22.634311    1515 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-20T19:24:22.642001950Z Wrote inspect data to must-gather.
2026-07-20T19:24:22.733662095Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:24:22.806704801Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:22.810000869Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-20T19:24:22.922474298Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-20T19:24:22.923198989Z Gathering data for ns/openshift-network-console...
2026-07-20T19:24:22.926390617Z INFO: Fallback to identify the container image from release info
2026-07-20T19:24:22.970000358Z INFO: Getting status from alertmanager-main-0
2026-07-20T19:24:23.014179982Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:23.217961624Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:23.220905311Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-20T19:24:23.338935188Z Gathering data for ns/openshift-console-operator...
2026-07-20T19:24:23.436724688Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:23.608515119Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:23.612082039Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-20T19:24:23.673911803Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-20T19:24:23.709880146Z Gathering data for ns/openshift-console...
2026-07-20T19:24:23.770128640Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:23.793167470Z daemonset.apps/perf-node-gather-daemonset created
2026-07-20T19:24:23.933827370Z Waiting for performance profile collector pods to become ready: 1
2026-07-20T19:24:23.964377959Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:24:24.247457545Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-20T19:24:24.617194649Z Done executing Istio gather script
2026-07-20T19:24:24.747718147Z Gathering data for ns/openshift-dns-operator...
2026-07-20T19:24:24.988731172Z Gathering data for ns/openshift-dns...
2026-07-20T19:24:25.038828655Z Waiting for performance profile collector pods to become ready: 2
2026-07-20T19:24:25.400019475Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-20T19:24:25.656154741Z Gathering data for ns/openshift-image-registry...
2026-07-20T19:24:26.088509223Z Gathering data for ns/openshift-ingress-operator...
2026-07-20T19:24:26.153969885Z Waiting for performance profile collector pods to become ready: 3
2026-07-20T19:24:26.389271306Z Gathering data for ns/openshift-ingress...
2026-07-20T19:24:26.671837613Z Gathering data for ns/openshift-ingress-canary...
2026-07-20T19:24:27.014269335Z Gathering data for ns/openshift-insights...
2026-07-20T19:24:27.253569115Z Waiting for performance profile collector pods to become ready: 4
2026-07-20T19:24:28.356679445Z Waiting for performance profile collector pods to become ready: 5
2026-07-20T19:24:28.806122135Z Gathering data for ns/openshift-jobset-operator...
2026-07-20T19:24:29.456985539Z Waiting for performance profile collector pods to become ready: 6
2026-07-20T19:24:29.546307125Z Gathering data for ns/openshift-config...
2026-07-20T19:24:29.784646684Z Gathering data for ns/openshift-config-managed...
2026-07-20T19:24:30.173009475Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-20T19:24:30.405107670Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-20T19:24:30.418015652Z Gathering data for ns/openshift-kube-apiserver...
2026-07-20T19:24:30.558377090Z Waiting for performance profile collector pods to become ready: 7
2026-07-20T19:24:30.711833563Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-20T19:24:31.011402855Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-20T19:24:31.343179257Z Gathering data for ns/openshift-kube-scheduler...
2026-07-20T19:24:31.606471899Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-20T19:24:31.678496086Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-20T19:24:31.909028181Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-20T19:24:31.911174674Z Collecting performance related data for node ip-10-0-133-155.ec2.internal
2026-07-20T19:24:31.917723901Z Collecting performance related data for node ip-10-0-135-182.ec2.internal
2026-07-20T19:24:31.924985584Z Collecting performance related data for node ip-10-0-130-194.ec2.internal
2026-07-20T19:24:32.313880228Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-20T19:24:32.667350345Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-20T19:24:33.268206605Z Gathering data for ns/openshift-multus...
2026-07-20T19:24:34.789489537Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-20T19:24:35.410440978Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-20T19:24:35.924325287Z Gathering data for ns/openshift-host-network...
2026-07-20T19:24:36.207451413Z Gathering data for ns/openshift-network-diagnostics...
2026-07-20T19:24:36.718127746Z Gathering data for ns/openshift-network-node-identity...
2026-07-20T19:24:37.092917662Z Gathering data for ns/openshift-network-operator...
2026-07-20T19:24:37.436881923Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-20T19:24:37.674242065Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-20T19:24:38.095397661Z Gathering data for ns/openshift-apiserver-operator...
2026-07-20T19:24:38.352883152Z Gathering data for ns/openshift-apiserver...
2026-07-20T19:24:38.710617703Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-20T19:24:39.015632265Z Gathering data for ns/openshift-controller-manager...
2026-07-20T19:24:39.299297691Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-20T19:24:39.791747786Z Collecting kubelet logs for node ip-10-0-133-155.ec2.internal
2026-07-20T19:24:39.793502930Z Collecting kubelet logs for node ip-10-0-135-182.ec2.internal
2026-07-20T19:24:39.795537355Z Collecting kubelet logs for node ip-10-0-130-194.ec2.internal
2026-07-20T19:24:39.927704257Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-20T19:24:40.114250302Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-smzlm namespace
2026-07-20T19:24:40.119706750Z INFO: Node performance data collection complete.
2026-07-20T19:24:40.219887159Z Gathering data for ns/openshift-service-ca-operator...
2026-07-20T19:24:40.418305813Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-20T19:24:40.529740051Z Gathering data for ns/openshift-service-ca...
2026-07-20T19:24:40.870223582Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-20T19:24:41.687607368Z Wrote inspect data to must-gather.
2026-07-20T19:24:41.687648418Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T19:24:41.687648418Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-20T19:24:41.802938391Z error: the server doesn't have a resource type "clusters"
2026-07-20T19:24:41.935319726Z Caches written to disk