2026-07-29T15:55:03.199664618Z [disk usage checker] Started
2026-07-29T15:55:03.203339697Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-07-29T15:55:03.408148166Z Gathering data for ns/openshift-cluster-version...
2026-07-29T15:55:03.550151643Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T15:55:03.644485647Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T15:55:03.732257162Z Gathering data for ns/default...
2026-07-29T15:55:04.035510308Z Gathering data for ns/openshift...
2026-07-29T15:55:04.462207240Z Gathering data for ns/kube-system...
2026-07-29T15:55:05.175214805Z Gathering data for ns/openshift-etcd...
2026-07-29T15:55:05.502221454Z Wrote inspect data to must-gather.
2026-07-29T15:55:05.502257325Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T15:55:05.502257325Z     namespaces "assisted-installer" not found
2026-07-29T15:55:05.865135280Z Waiting on subprocesses to finish execution.
2026-07-29T15:55:05.871106781Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-29T15:55:05.872571364Z Executing Istio gather script
2026-07-29T15:55:05.875915528Z INFO: Gathering HAProxy config files
2026-07-29T15:55:05.881635494Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-29T15:55:05.886583317Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-29T15:55:05.888699297Z INFO: Collecting host service logs for crio
2026-07-29T15:55:05.889051968Z INFO: Collecting host service logs for kubelet
2026-07-29T15:55:05.889360070Z INFO: Collecting host service logs for rpm-ostreed
2026-07-29T15:55:05.889699511Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-29T15:55:05.890016046Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-29T15:55:05.890328969Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-29T15:55:05.890639762Z INFO: Collecting host service logs for NetworkManager
2026-07-29T15:55:05.890962879Z INFO: Collecting host service logs for openvswitch
2026-07-29T15:55:05.891266124Z INFO: Collecting host service logs for ovs-configuration
2026-07-29T15:55:05.891596674Z INFO: Collecting host service logs for ovsdb-server
2026-07-29T15:55:05.891909817Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-29T15:55:05.892291315Z INFO: Waiting for worker host service log collection to complete ...
2026-07-29T15:55:05.974573881Z INFO: Waiting for node performance related collection to complete ...
2026-07-29T15:55:06.328631095Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-ttnzr
2026-07-29T15:55:06.328711249Z insights-runtime-extractor-2rv6w
2026-07-29T15:55:06.328731892Z insights-runtime-extractor-lgb2h
2026-07-29T15:55:06.328760316Z insights-runtime-extractor-mn2h4
2026-07-29T15:55:06.557601634Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-29T15:55:06.762521462Z Wrote inspect data to must-gather.
2026-07-29T15:55:06.852078415Z error: the server doesn't have a resource type "performanceprofile"
2026-07-29T15:55:06.978983144Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-29T15:55:07.001524195Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-29T15:55:07.231607492Z INFO: "metallb-operator" not detected. Skipping.
2026-07-29T15:55:07.236506933Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-29T15:55:07.269466584Z No resources found
2026-07-29T15:55:07.285098439Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-29T15:55:07.285188181Z INFO: on-disk MachineConfig config collection complete.
2026-07-29T15:55:07.305017882Z No resources found in openshift-etcd namespace.
2026-07-29T15:55:07.430226974Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-29T15:55:07.430226974Z See 'oc rsync -h' for help and examples
2026-07-29T15:55:07.447962710Z INFO: Worker host service log collection to complete.
2026-07-29T15:55:07.625546810Z ERROR: No running kube-apiserver pods found
2026-07-29T15:55:07.653834300Z Wrote inspect data to must-gather.
2026-07-29T15:55:07.673012832Z Wrote inspect data to must-gather.
2026-07-29T15:55:07.731051169Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-29T15:55:07.916593897Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-29T15:55:08.056749879Z Wrote inspect data to must-gather.
2026-07-29T15:55:08.080650929Z No resources found
2026-07-29T15:55:08.143146753Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-29T15:55:08.195611264Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T15:55:08.229181709Z Wrote inspect data to must-gather.
2026-07-29T15:55:08.235989121Z error: the server doesn't have a resource type "machineconfigs"
2026-07-29T15:55:08.267960412Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-07-29T15:55:08.445989500Z Wrote inspect data to must-gather.
2026-07-29T15:55:08.501341413Z INFO: 'previous-logs' folder not found on ip-10-0-136-4.ec2.internal, skipping...
2026-07-29T15:55:08.523218639Z Wrote inspect data to must-gather.
2026-07-29T15:55:08.585739398Z tar: Removing leading `/' from member names
2026-07-29T15:55:08.593953477Z INFO: HAProxy config collection complete.
2026-07-29T15:55:08.607743545Z Gathering data for ns/openshift-monitoring...
2026-07-29T15:55:08.616842371Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-29T15:55:08.625357675Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-29T15:55:08.800270816Z INFO: 'previous-logs' folder not found on ip-10-0-137-65.ec2.internal, skipping...
2026-07-29T15:55:08.821508833Z INFO: Getting rules from prometheus-k8s-0
2026-07-29T15:55:08.822552623Z Wrote inspect data to must-gather.
2026-07-29T15:55:08.854147856Z Wrote inspect data to must-gather.
2026-07-29T15:55:09.035074623Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-29T15:55:09.091095712Z INFO: 'previous-logs' folder not found on ip-10-0-142-124.ec2.internal, skipping...
2026-07-29T15:55:09.091201297Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-29T15:55:09.091351600Z INFO: Machine Config Daemon termination log collection complete.
2026-07-29T15:55:09.092382549Z Wrote inspect data to must-gather.
2026-07-29T15:55:09.253323643Z INFO: Getting status/config from prometheus-k8s-0
2026-07-29T15:55:09.269203000Z Wrote inspect data to must-gather.
2026-07-29T15:55:09.355372257Z Wrote inspect data to must-gather.
2026-07-29T15:55:09.479578950Z Wrote inspect data to must-gather.
2026-07-29T15:55:09.645652618Z INFO: INTERCONNECT MODE
2026-07-29T15:55:09.645696924Z INFO: Gathering ovn-kubernetes DBs
2026-07-29T15:55:09.658353398Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-29T15:55:09.687602137Z Wrote inspect data to must-gather.
2026-07-29T15:55:09.781015280Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-29T15:55:09.819046443Z INFO: Gathering OVN_Northbound from ovnkube-node-c6xx5...
2026-07-29T15:55:09.819393796Z INFO: Gathering OVN_Northbound from ovnkube-node-g7svn...
2026-07-29T15:55:09.819668410Z INFO: Gathering OVN_Northbound from ovnkube-node-kl8gn...
2026-07-29T15:55:09.819919281Z INFO: Gathering OVN_Southbound from ovnkube-node-c6xx5...
2026-07-29T15:55:09.820165515Z INFO: Gathering OVN_Southbound from ovnkube-node-g7svn...
2026-07-29T15:55:09.820445488Z INFO: Gathering OVN_Southbound from ovnkube-node-kl8gn...
2026-07-29T15:55:09.976574500Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-29T15:55:09.989566275Z Wrote inspect data to must-gather.
2026-07-29T15:55:10.088309974Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:10.129439401Z tar: Removing leading `/' from member names
2026-07-29T15:55:10.158154987Z tar: Removing leading `/' from member names
2026-07-29T15:55:10.190896432Z tar: Removing leading `/' from member names
2026-07-29T15:55:10.195428195Z tar: Removing leading `/' from member names
2026-07-29T15:55:10.207248617Z tar: Removing leading `/' from member names
2026-07-29T15:55:10.216153735Z tar: Removing leading `/' from member names
2026-07-29T15:55:10.289586422Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-29T15:55:10.337906008Z Wrote inspect data to must-gather.
2026-07-29T15:55:10.341523417Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-29T15:55:10.366403887Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:10.369404366Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-29T15:55:10.597282654Z Wrote inspect data to must-gather.
2026-07-29T15:55:10.623904146Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:10.647188454Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-29T15:55:10.757817348Z INFO: Waiting for network log collection to complete ...
2026-07-29T15:55:10.757890372Z INFO: Waiting for ovnk database copies to complete ...
2026-07-29T15:55:10.758419021Z INFO: Copying ovnk databases complete.
2026-07-29T15:55:10.760174322Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-29T15:55:10.762438190Z ovnk_database_store/
2026-07-29T15:55:10.762438190Z ovnk_database_store/ovnkube-node-c6xx5_nbdb
2026-07-29T15:55:10.774412051Z ovnk_database_store/ovnkube-node-c6xx5_sbdb
2026-07-29T15:55:10.796146591Z ovnk_database_store/ovnkube-node-kl8gn_nbdb
2026-07-29T15:55:10.802780467Z ovnk_database_store/ovnkube-node-g7svn_nbdb
2026-07-29T15:55:10.809501160Z ovnk_database_store/ovnkube-node-g7svn_sbdb
2026-07-29T15:55:10.829264795Z ovnk_database_store/ovnkube-node-kl8gn_sbdb
2026-07-29T15:55:10.843672068Z W0729 15:55:10.843622    1512 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-29T15:55:10.848567181Z INFO: Network log collection complete.
2026-07-29T15:55:10.851736672Z Wrote inspect data to must-gather.
2026-07-29T15:55:10.864127171Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:10.866765252Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-29T15:55:10.993120580Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-29T15:55:11.084898587Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:11.087698320Z INFO: Getting status from alertmanager-main-0
2026-07-29T15:55:11.146201367Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-29T15:55:11.150970158Z INFO: Fallback to identify the container image from release info
2026-07-29T15:55:11.202622422Z Gathering data for ns/openshift-network-console...
2026-07-29T15:55:11.313220100Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:11.315825965Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-29T15:55:11.532097463Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:11.645322702Z Gathering data for ns/openshift-console-operator...
2026-07-29T15:55:11.733256258Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:11.737685552Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-29T15:55:11.825567298Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-29T15:55:11.909222018Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:11.962152352Z daemonset.apps/perf-node-gather-daemonset created
2026-07-29T15:55:12.074170462Z Gathering data for ns/openshift-console...
2026-07-29T15:55:12.100293183Z Waiting for performance profile collector pods to become ready: 1
2026-07-29T15:55:12.126329164Z Wrote inspect data to /must-gather/istio.
2026-07-29T15:55:12.591579692Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-29T15:55:12.815349320Z Done executing Istio gather script
2026-07-29T15:55:13.110030153Z Gathering data for ns/openshift-dns-operator...
2026-07-29T15:55:13.226981698Z Waiting for performance profile collector pods to become ready: 2
2026-07-29T15:55:13.277595346Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-07-29T15:55:13.446785028Z Gathering data for ns/openshift-dns...
2026-07-29T15:55:14.111980634Z Gathering data for ns/openshift-image-registry...
2026-07-29T15:55:14.348473948Z Waiting for performance profile collector pods to become ready: 3
2026-07-29T15:55:14.575536840Z Gathering data for ns/openshift-ingress-operator...
2026-07-29T15:55:14.913797751Z Gathering data for ns/openshift-ingress...
2026-07-29T15:55:15.252201039Z Gathering data for ns/openshift-ingress-canary...
2026-07-29T15:55:15.463568784Z Waiting for performance profile collector pods to become ready: 4
2026-07-29T15:55:15.635482913Z Gathering data for ns/openshift-insights...
2026-07-29T15:55:16.567894114Z Waiting for performance profile collector pods to become ready: 5
2026-07-29T15:55:17.445481381Z Gathering data for ns/openshift-jobset-operator...
2026-07-29T15:55:17.686057600Z Waiting for performance profile collector pods to become ready: 6
2026-07-29T15:55:18.221669715Z Gathering data for ns/openshift-config...
2026-07-29T15:55:18.283526289Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-07-29T15:55:18.478667643Z Gathering data for ns/openshift-config-managed...
2026-07-29T15:55:18.798422714Z Waiting for performance profile collector pods to become ready: 7
2026-07-29T15:55:18.903518711Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-29T15:55:19.173064195Z Gathering data for ns/openshift-kube-apiserver...
2026-07-29T15:55:19.444357221Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-29T15:55:19.787331099Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-29T15:55:19.915687966Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-29T15:55:20.100177583Z Gathering data for ns/openshift-kube-scheduler...
2026-07-29T15:55:20.194735058Z Collecting performance related data for node ip-10-0-136-4.ec2.internal
2026-07-29T15:55:20.202286765Z Collecting performance related data for node ip-10-0-142-124.ec2.internal
2026-07-29T15:55:20.210274952Z Collecting performance related data for node ip-10-0-137-65.ec2.internal
2026-07-29T15:55:20.408454127Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-29T15:55:20.749867411Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-29T15:55:21.172246194Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-29T15:55:21.575219434Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-29T15:55:22.107013859Z Gathering data for ns/openshift-multus...
2026-07-29T15:55:23.290018012Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-07-29T15:55:23.629074720Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-29T15:55:25.287475278Z Gathering data for ns/openshift-host-network...
2026-07-29T15:55:25.609883820Z Gathering data for ns/openshift-network-diagnostics...
2026-07-29T15:55:26.227403255Z Gathering data for ns/openshift-network-node-identity...
2026-07-29T15:55:26.652857212Z Gathering data for ns/openshift-network-operator...
2026-07-29T15:55:27.062087990Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-29T15:55:27.324132158Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-29T15:55:27.800465899Z Gathering data for ns/openshift-apiserver-operator...
2026-07-29T15:55:28.096393391Z Gathering data for ns/openshift-apiserver...
2026-07-29T15:55:28.263678769Z Collecting kubelet logs for node ip-10-0-136-4.ec2.internal
2026-07-29T15:55:28.265633473Z Collecting kubelet logs for node ip-10-0-142-124.ec2.internal
2026-07-29T15:55:28.268154530Z Collecting kubelet logs for node ip-10-0-137-65.ec2.internal
2026-07-29T15:55:28.297277401Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70
2026-07-29T15:55:28.553569243Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-29T15:55:28.640264589Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-g66fj namespace
2026-07-29T15:55:28.645965738Z INFO: Node performance data collection complete.
2026-07-29T15:55:28.877194854Z Gathering data for ns/openshift-controller-manager...
2026-07-29T15:55:29.183259006Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-29T15:55:29.822837588Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-29T15:55:30.105232386Z Gathering data for ns/openshift-service-ca-operator...
2026-07-29T15:55:30.428315875Z Gathering data for ns/openshift-service-ca...
2026-07-29T15:55:30.775281452Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-29T15:55:31.699788009Z Wrote inspect data to must-gather.
2026-07-29T15:55:31.699833940Z error: inspection completed with the errors occurred while gathering data:
2026-07-29T15:55:31.699833940Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-29T15:55:31.822719277Z error: the server doesn't have a resource type "clusters"
2026-07-29T15:55:31.955694377Z Caches written to disk