2026-07-28T14:16:25.141416302Z [disk usage checker] Started
2026-07-28T14:16:25.144743934Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-28T14:16:25.341675346Z Gathering data for ns/openshift-cluster-version...
2026-07-28T14:16:25.474879858Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T14:16:25.570064762Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T14:16:25.653439729Z Gathering data for ns/default...
2026-07-28T14:16:25.963651010Z Gathering data for ns/openshift...
2026-07-28T14:16:26.390374131Z Gathering data for ns/kube-system...
2026-07-28T14:16:27.273984393Z Gathering data for ns/openshift-etcd...
2026-07-28T14:16:27.589208402Z Wrote inspect data to must-gather.
2026-07-28T14:16:27.589240270Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T14:16:27.589240270Z     namespaces "assisted-installer" not found
2026-07-28T14:16:27.698999528Z Waiting on subprocesses to finish execution.
2026-07-28T14:16:27.709627616Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-28T14:16:27.709858930Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-28T14:16:27.712256356Z INFO: Collecting host service logs for crio
2026-07-28T14:16:27.712654422Z INFO: Collecting host service logs for kubelet
2026-07-28T14:16:27.713009961Z INFO: Collecting host service logs for rpm-ostreed
2026-07-28T14:16:27.713375678Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-28T14:16:27.713759546Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-28T14:16:27.714089184Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-28T14:16:27.714420913Z INFO: Collecting host service logs for NetworkManager
2026-07-28T14:16:27.714774609Z INFO: Collecting host service logs for openvswitch
2026-07-28T14:16:27.715061166Z INFO: Gathering HAProxy config files
2026-07-28T14:16:27.715131477Z INFO: Collecting host service logs for ovs-configuration
2026-07-28T14:16:27.715467020Z INFO: Collecting host service logs for ovsdb-server
2026-07-28T14:16:27.715818478Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-28T14:16:27.716218884Z INFO: Waiting for worker host service log collection to complete ...
2026-07-28T14:16:27.717918698Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-28T14:16:27.775578533Z Executing Istio gather script
2026-07-28T14:16:28.006179084Z INFO: Waiting for node performance related collection to complete ...
2026-07-28T14:16:28.252778605Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-28T14:16:28.252861707Z INFO: on-disk MachineConfig config collection complete.
2026-07-28T14:16:28.444740423Z INFO: "metallb-operator" not detected. Skipping.
2026-07-28T14:16:28.463746391Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-28T14:16:28.608550974Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-6pngg
2026-07-28T14:16:28.608597877Z insights-runtime-extractor-l52wv
2026-07-28T14:16:28.608617639Z insights-runtime-extractor-lxwmm
2026-07-28T14:16:28.608637272Z insights-runtime-extractor-w4qn7
2026-07-28T14:16:28.637492744Z Wrote inspect data to must-gather.
2026-07-28T14:16:28.637696170Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-28T14:16:28.826163153Z No resources found in openshift-etcd namespace.
2026-07-28T14:16:28.846743239Z No resources found
2026-07-28T14:16:28.938321999Z error: the server doesn't have a resource type "performanceprofile"
2026-07-28T14:16:28.950114281Z INFO: Worker host service log collection to complete.
2026-07-28T14:16:29.128400374Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-28T14:16:29.128400374Z See 'oc rsync -h' for help and examples
2026-07-28T14:16:29.137397219Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-28T14:16:29.209335602Z ERROR: No running kube-apiserver pods found
2026-07-28T14:16:29.226298746Z No resources found
2026-07-28T14:16:29.238507248Z Wrote inspect data to must-gather.
2026-07-28T14:16:29.306715660Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-28T14:16:29.337923396Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T14:16:29.425724729Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-28T14:16:29.511911078Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-28T14:16:29.727705519Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-28T14:16:29.784079533Z Wrote inspect data to must-gather.
2026-07-28T14:16:29.841835398Z Wrote inspect data to must-gather.
2026-07-28T14:16:29.871077812Z Wrote inspect data to must-gather.
2026-07-28T14:16:29.925135158Z Wrote inspect data to must-gather.
2026-07-28T14:16:29.950788574Z INFO: 'previous-logs' folder not found on ip-10-0-130-220.ec2.internal, skipping...
2026-07-28T14:16:29.975243177Z Wrote inspect data to must-gather.
2026-07-28T14:16:30.037948765Z tar: Removing leading `/' from member names
2026-07-28T14:16:30.044489078Z INFO: HAProxy config collection complete.
2026-07-28T14:16:30.080568305Z error: the server doesn't have a resource type "machineconfigs"
2026-07-28T14:16:30.156345900Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-28T14:16:30.233239398Z Wrote inspect data to must-gather.
2026-07-28T14:16:30.254490462Z INFO: Getting rules from prometheus-k8s-0
2026-07-28T14:16:30.270457063Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-28T14:16:30.290651944Z INFO: 'previous-logs' folder not found on ip-10-0-131-116.ec2.internal, skipping...
2026-07-28T14:16:30.309783751Z Gathering data for ns/openshift-monitoring...
2026-07-28T14:16:30.516967358Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-28T14:16:30.531645370Z Wrote inspect data to must-gather.
2026-07-28T14:16:30.561897262Z Wrote inspect data to must-gather.
2026-07-28T14:16:30.577311128Z INFO: 'previous-logs' folder not found on ip-10-0-136-179.ec2.internal, skipping...
2026-07-28T14:16:30.577345085Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-28T14:16:30.577378127Z INFO: Machine Config Daemon termination log collection complete.
2026-07-28T14:16:30.700835605Z INFO: Getting status/config from prometheus-k8s-0
2026-07-28T14:16:30.707567215Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-28T14:16:30.803866738Z Wrote inspect data to must-gather.
2026-07-28T14:16:30.910656250Z Wrote inspect data to must-gather.
2026-07-28T14:16:30.961806562Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-28T14:16:31.057711719Z INFO: INTERCONNECT MODE
2026-07-28T14:16:31.057825747Z INFO: Gathering ovn-kubernetes DBs
2026-07-28T14:16:31.106052510Z Wrote inspect data to must-gather.
2026-07-28T14:16:31.157923526Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:31.182978697Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-28T14:16:31.227781862Z INFO: Gathering OVN_Northbound from ovnkube-node-l5cdj...
2026-07-28T14:16:31.228039841Z INFO: Gathering OVN_Northbound from ovnkube-node-nqtlw...
2026-07-28T14:16:31.228338794Z INFO: Gathering OVN_Northbound from ovnkube-node-nx7bb...
2026-07-28T14:16:31.228662198Z INFO: Gathering OVN_Southbound from ovnkube-node-l5cdj...
2026-07-28T14:16:31.228964178Z INFO: Gathering OVN_Southbound from ovnkube-node-nqtlw...
2026-07-28T14:16:31.229247918Z INFO: Gathering OVN_Southbound from ovnkube-node-nx7bb...
2026-07-28T14:16:31.429146869Z Wrote inspect data to must-gather.
2026-07-28T14:16:31.527730901Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:31.530656789Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-28T14:16:31.652785571Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-28T14:16:31.656683590Z tar: Removing leading `/' from member names
2026-07-28T14:16:31.656883068Z Wrote inspect data to must-gather.
2026-07-28T14:16:31.686276981Z tar: Removing leading `/' from member names
2026-07-28T14:16:31.688015084Z tar: Removing leading `/' from member names
2026-07-28T14:16:31.706309706Z tar: Removing leading `/' from member names
2026-07-28T14:16:31.710057617Z tar: Removing leading `/' from member names
2026-07-28T14:16:31.711716295Z tar: Removing leading `/' from member names
2026-07-28T14:16:31.756521461Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:31.888163107Z Wrote inspect data to must-gather.
2026-07-28T14:16:31.954651288Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-28T14:16:31.978682595Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:31.982137617Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-28T14:16:32.016377179Z INFO: Waiting for network log collection to complete ...
2026-07-28T14:16:32.016407433Z INFO: Waiting for ovnk database copies to complete ...
2026-07-28T14:16:32.016820628Z INFO: Copying ovnk databases complete.
2026-07-28T14:16:32.019119598Z 3.6M	must-gather/network_logs/ovnk_database_store
2026-07-28T14:16:32.021926013Z ovnk_database_store/
2026-07-28T14:16:32.021971810Z ovnk_database_store/ovnkube-node-nx7bb_nbdb
2026-07-28T14:16:32.035795399Z ovnk_database_store/ovnkube-node-nqtlw_sbdb
2026-07-28T14:16:32.056392469Z ovnk_database_store/ovnkube-node-nqtlw_nbdb
2026-07-28T14:16:32.064440594Z ovnk_database_store/ovnkube-node-l5cdj_nbdb
2026-07-28T14:16:32.072316431Z ovnk_database_store/ovnkube-node-l5cdj_sbdb
2026-07-28T14:16:32.087405499Z Wrote inspect data to must-gather.
2026-07-28T14:16:32.093729195Z ovnk_database_store/ovnkube-node-nx7bb_sbdb
2026-07-28T14:16:32.119166529Z INFO: Network log collection complete.
2026-07-28T14:16:32.198645667Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:32.275328659Z Wrote inspect data to must-gather.
2026-07-28T14:16:32.294574082Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-28T14:16:32.383708775Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:32.386780386Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-28T14:16:32.513791851Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-28T14:16:32.515824358Z INFO: Fallback to identify the container image from release info
2026-07-28T14:16:32.609778040Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:32.714175902Z INFO: Getting status from alertmanager-main-0
2026-07-28T14:16:32.761190863Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T14:16:32.810992344Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:32.814188909Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-28T14:16:32.961139088Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:33.143185200Z Wrote inspect data to /must-gather/istio.
2026-07-28T14:16:33.254430842Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-28T14:16:33.316299198Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T14:16:33.398355210Z daemonset.apps/perf-node-gather-daemonset created
2026-07-28T14:16:33.505418965Z Gathering data for ns/openshift-network-console...
2026-07-28T14:16:33.532617991Z Waiting for performance profile collector pods to become ready: 1
2026-07-28T14:16:33.822228041Z Done executing Istio gather script
2026-07-28T14:16:33.945837285Z Gathering data for ns/openshift-console-operator...
2026-07-28T14:16:34.335224843Z Gathering data for ns/openshift-console...
2026-07-28T14:16:34.639406021Z Waiting for performance profile collector pods to become ready: 2
2026-07-28T14:16:34.838353208Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-28T14:16:35.161610727Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-28T14:16:35.344436052Z Gathering data for ns/openshift-dns-operator...
2026-07-28T14:16:35.634151444Z Gathering data for ns/openshift-dns...
2026-07-28T14:16:35.742066273Z Waiting for performance profile collector pods to become ready: 3
2026-07-28T14:16:36.276309489Z Gathering data for ns/openshift-image-registry...
2026-07-28T14:16:36.810819101Z Gathering data for ns/openshift-ingress-operator...
2026-07-28T14:16:36.867774407Z Waiting for performance profile collector pods to become ready: 4
2026-07-28T14:16:37.177113875Z Gathering data for ns/openshift-ingress...
2026-07-28T14:16:37.528009801Z Gathering data for ns/openshift-ingress-canary...
2026-07-28T14:16:37.931717643Z Gathering data for ns/openshift-insights...
2026-07-28T14:16:37.982606230Z Waiting for performance profile collector pods to become ready: 5
2026-07-28T14:16:39.113271138Z Waiting for performance profile collector pods to become ready: 6
2026-07-28T14:16:39.927615271Z Gathering data for ns/openshift-config...
2026-07-28T14:16:40.167805336Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-28T14:16:40.230333018Z Waiting for performance profile collector pods to become ready: 7
2026-07-28T14:16:40.280383182Z Gathering data for ns/openshift-config-managed...
2026-07-28T14:16:40.757204312Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-28T14:16:41.084989873Z Gathering data for ns/openshift-kube-apiserver...
2026-07-28T14:16:41.350839881Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-28T14:16:41.386389854Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-28T14:16:41.619391611Z Collecting performance related data for node ip-10-0-131-116.ec2.internal
2026-07-28T14:16:41.626382654Z Collecting performance related data for node ip-10-0-136-179.ec2.internal
2026-07-28T14:16:41.635023724Z Collecting performance related data for node ip-10-0-130-220.ec2.internal
2026-07-28T14:16:41.762878804Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-28T14:16:42.102459532Z Gathering data for ns/openshift-kube-scheduler...
2026-07-28T14:16:42.466914738Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-28T14:16:42.807253534Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-28T14:16:43.180984562Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-28T14:16:43.480594338Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-28T14:16:43.999990748Z Gathering data for ns/openshift-multus...
2026-07-28T14:16:45.172528944Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-28T14:16:45.514108470Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-28T14:16:46.974341409Z Gathering data for ns/openshift-host-network...
2026-07-28T14:16:47.340216256Z Gathering data for ns/openshift-network-diagnostics...
2026-07-28T14:16:47.896394731Z Gathering data for ns/openshift-network-node-identity...
2026-07-28T14:16:48.322795976Z Gathering data for ns/openshift-network-operator...
2026-07-28T14:16:48.766554467Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-28T14:16:49.099259857Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-28T14:16:49.273014448Z Collecting kubelet logs for node ip-10-0-131-116.ec2.internal
2026-07-28T14:16:49.274798700Z Collecting kubelet logs for node ip-10-0-136-179.ec2.internal
2026-07-28T14:16:49.276457754Z Collecting kubelet logs for node ip-10-0-130-220.ec2.internal
2026-07-28T14:16:49.633703292Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-hn42g namespace
2026-07-28T14:16:49.640245534Z INFO: Node performance data collection complete.
2026-07-28T14:16:49.667657897Z Gathering data for ns/openshift-apiserver-operator...
2026-07-28T14:16:50.041498971Z Gathering data for ns/openshift-apiserver...
2026-07-28T14:16:50.177305327Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-28T14:16:50.506455478Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-28T14:16:50.826525316Z Gathering data for ns/openshift-controller-manager...
2026-07-28T14:16:51.213693910Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-28T14:16:51.965352599Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-28T14:16:52.243730950Z Gathering data for ns/openshift-service-ca-operator...
2026-07-28T14:16:52.580576740Z Gathering data for ns/openshift-service-ca...
2026-07-28T14:16:52.946352189Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-28T14:16:53.863353987Z Wrote inspect data to must-gather.
2026-07-28T14:16:53.863405783Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T14:16:53.863405783Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-28T14:16:53.983901665Z error: the server doesn't have a resource type "clusters"
2026-07-28T14:16:54.119531183Z Caches written to disk