2026-07-21T07:58:21.446594579Z [disk usage checker] Started
2026-07-21T07:58:21.449861017Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-21T07:58:21.641693325Z Gathering data for ns/openshift-cluster-version...
2026-07-21T07:58:21.780609084Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-21T07:58:21.858701613Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T07:58:21.936709299Z Gathering data for ns/default...
2026-07-21T07:58:22.206192555Z Gathering data for ns/openshift...
2026-07-21T07:58:22.587373167Z Gathering data for ns/kube-system...
2026-07-21T07:58:23.225245981Z Gathering data for ns/openshift-etcd...
2026-07-21T07:58:23.509695350Z Wrote inspect data to must-gather.
2026-07-21T07:58:23.509730069Z error: inspection completed with the errors occurred while gathering data:
2026-07-21T07:58:23.509730069Z     namespaces "assisted-installer" not found
2026-07-21T07:58:23.886280453Z Waiting on subprocesses to finish execution.
2026-07-21T07:58:23.893372205Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-21T07:58:23.901285251Z INFO: Gathering HAProxy config files
2026-07-21T07:58:23.902932644Z Executing Istio gather script
2026-07-21T07:58:23.903431141Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-21T07:58:23.905508859Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-21T07:58:23.997571505Z INFO: Collecting host service logs for crio
2026-07-21T07:58:23.997838989Z INFO: Collecting host service logs for kubelet
2026-07-21T07:58:23.998147299Z INFO: Collecting host service logs for rpm-ostreed
2026-07-21T07:58:23.998485931Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-21T07:58:23.998758534Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-21T07:58:23.999041582Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-21T07:58:23.999340010Z INFO: Collecting host service logs for NetworkManager
2026-07-21T07:58:23.999630842Z INFO: Collecting host service logs for openvswitch
2026-07-21T07:58:23.999897692Z INFO: Collecting host service logs for ovs-configuration
2026-07-21T07:58:24.000174499Z INFO: Collecting host service logs for ovsdb-server
2026-07-21T07:58:24.000451469Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-21T07:58:24.000774639Z INFO: Waiting for worker host service log collection to complete ...
2026-07-21T07:58:24.209316744Z INFO: Waiting for node performance related collection to complete ...
2026-07-21T07:58:24.451156236Z No resources found in openshift-etcd namespace.
2026-07-21T07:58:24.561581311Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-6zh6q
2026-07-21T07:58:24.561635378Z insights-runtime-extractor-82xbn
2026-07-21T07:58:24.561655609Z insights-runtime-extractor-sbwtb
2026-07-21T07:58:24.561674858Z insights-runtime-extractor-zhb67
2026-07-21T07:58:24.662591623Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-21T07:58:24.669313229Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-21T07:58:24.724065404Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-21T07:58:24.724104124Z INFO: on-disk MachineConfig config collection complete.
2026-07-21T07:58:24.728246111Z Wrote inspect data to must-gather.
2026-07-21T07:58:25.048985257Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-21T07:58:25.230743502Z INFO: "metallb-operator" not detected. Skipping.
2026-07-21T07:58:25.305576800Z error: the server doesn't have a resource type "performanceprofile"
2026-07-21T07:58:25.320482228Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-21T07:58:25.365092593Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-21T07:58:25.365092593Z See 'oc rsync -h' for help and examples
2026-07-21T07:58:25.446236172Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-21T07:58:25.559565320Z ERROR: No running kube-apiserver pods found
2026-07-21T07:58:25.577166615Z No resources found
2026-07-21T07:58:25.746863900Z INFO: 'previous-logs' folder not found on ip-10-0-130-251.ec2.internal, skipping...
2026-07-21T07:58:25.822088513Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-21T07:58:25.836015106Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-21T07:58:25.847103056Z No resources found
2026-07-21T07:58:26.000704266Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.061758011Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.109557568Z tar: Removing leading `/' from member names
2026-07-21T07:58:26.115210142Z INFO: HAProxy config collection complete.
2026-07-21T07:58:26.171019127Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T07:58:26.235214418Z INFO: Worker host service log collection to complete.
2026-07-21T07:58:26.267790120Z Gathering data for ns/openshift-monitoring...
2026-07-21T07:58:26.298246561Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.322120756Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.451868522Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.456566979Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-21T07:58:26.534301324Z INFO: 'previous-logs' folder not found on ip-10-0-137-224.ec2.internal, skipping...
2026-07-21T07:58:26.544299719Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.603734716Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.701255807Z INFO: Getting rules from prometheus-k8s-0
2026-07-21T07:58:26.723180692Z error: the server doesn't have a resource type "machineconfigs"
2026-07-21T07:58:26.842886013Z INFO: 'previous-logs' folder not found on ip-10-0-142-172.ec2.internal, skipping...
2026-07-21T07:58:26.842993644Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-21T07:58:26.843048302Z INFO: Machine Config Daemon termination log collection complete.
2026-07-21T07:58:26.900584665Z Wrote inspect data to must-gather.
2026-07-21T07:58:26.910020801Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-21T07:58:26.913847352Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-21T07:58:27.118363138Z Wrote inspect data to must-gather.
2026-07-21T07:58:27.127595668Z INFO: Getting status/config from prometheus-k8s-0
2026-07-21T07:58:27.144664877Z Wrote inspect data to must-gather.
2026-07-21T07:58:27.273041996Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-21T07:58:27.430146893Z INFO: INTERCONNECT MODE
2026-07-21T07:58:27.430176252Z INFO: Gathering ovn-kubernetes DBs
2026-07-21T07:58:27.491941725Z Wrote inspect data to must-gather.
2026-07-21T07:58:27.495198375Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-21T07:58:27.597498756Z INFO: Gathering OVN_Northbound from ovnkube-node-4cz9z...
2026-07-21T07:58:27.597784457Z INFO: Gathering OVN_Northbound from ovnkube-node-9g7t6...
2026-07-21T07:58:27.598047992Z INFO: Gathering OVN_Northbound from ovnkube-node-vg4pb...
2026-07-21T07:58:27.598341981Z INFO: Gathering OVN_Southbound from ovnkube-node-4cz9z...
2026-07-21T07:58:27.598654501Z INFO: Gathering OVN_Southbound from ovnkube-node-9g7t6...
2026-07-21T07:58:27.598906347Z INFO: Gathering OVN_Southbound from ovnkube-node-vg4pb...
2026-07-21T07:58:27.776241743Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-21T07:58:27.780273047Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-21T07:58:27.789983969Z Wrote inspect data to must-gather.
2026-07-21T07:58:27.873320726Z tar: Removing leading `/' from member names
2026-07-21T07:58:27.917303887Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-21T07:58:27.946550639Z tar: Removing leading `/' from member names
2026-07-21T07:58:27.953486416Z tar: Removing leading `/' from member names
2026-07-21T07:58:27.963228724Z tar: Removing leading `/' from member names
2026-07-21T07:58:27.979177865Z tar: Removing leading `/' from member names
2026-07-21T07:58:28.002807209Z tar: Removing leading `/' from member names
2026-07-21T07:58:28.035803996Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:28.091401599Z Wrote inspect data to must-gather.
2026-07-21T07:58:28.133174230Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-21T07:58:28.288542810Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:28.290774099Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-21T07:58:28.348459588Z Wrote inspect data to must-gather.
2026-07-21T07:58:28.408501636Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-21T07:58:28.455165929Z INFO: Waiting for network log collection to complete ...
2026-07-21T07:58:28.455215230Z INFO: Waiting for ovnk database copies to complete ...
2026-07-21T07:58:28.455726017Z INFO: Copying ovnk databases complete.
2026-07-21T07:58:28.457283769Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-21T07:58:28.459658685Z ovnk_database_store/
2026-07-21T07:58:28.459658685Z ovnk_database_store/ovnkube-node-vg4pb_nbdb
2026-07-21T07:58:28.468005156Z ovnk_database_store/ovnkube-node-9g7t6_sbdb
2026-07-21T07:58:28.487284979Z ovnk_database_store/ovnkube-node-4cz9z_sbdb
2026-07-21T07:58:28.506944602Z ovnk_database_store/ovnkube-node-4cz9z_nbdb
2026-07-21T07:58:28.513267590Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:28.513851927Z ovnk_database_store/ovnkube-node-9g7t6_nbdb
2026-07-21T07:58:28.520588599Z ovnk_database_store/ovnkube-node-vg4pb_sbdb
2026-07-21T07:58:28.541433485Z INFO: Network log collection complete.
2026-07-21T07:58:28.567090498Z W0721 07:58:28.567043    1494 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-21T07:58:28.574735393Z Wrote inspect data to must-gather.
2026-07-21T07:58:28.614583994Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T07:58:28.738966332Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:28.741770177Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-21T07:58:28.796871624Z INFO: Getting status from alertmanager-main-0
2026-07-21T07:58:28.840279075Z Gathering data for ns/openshift-network-console...
2026-07-21T07:58:28.884531908Z Wrote inspect data to must-gather.
2026-07-21T07:58:28.939892941Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:29.144816973Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:29.147125124Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-21T07:58:29.192742454Z Wrote inspect data to must-gather.
2026-07-21T07:58:29.347316410Z Gathering data for ns/openshift-console-operator...
2026-07-21T07:58:29.374757603Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:29.445826218Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-21T07:58:29.449126108Z INFO: Fallback to identify the container image from release info
2026-07-21T07:58:29.576227303Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:29.578539197Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-21T07:58:29.736799016Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:29.801927512Z Gathering data for ns/openshift-console...
2026-07-21T07:58:29.910514235Z Wrote inspect data to /must-gather/istio.
2026-07-21T07:58:30.261722421Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-21T07:58:30.383138749Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-21T07:58:30.390830804Z daemonset.apps/perf-node-gather-daemonset created
2026-07-21T07:58:30.524444682Z Waiting for performance profile collector pods to become ready: 1
2026-07-21T07:58:30.578796333Z Done executing Istio gather script
2026-07-21T07:58:30.916671532Z Gathering data for ns/openshift-dns-operator...
2026-07-21T07:58:31.233889016Z Gathering data for ns/openshift-dns...
2026-07-21T07:58:31.463079324Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-21T07:58:31.634185697Z Waiting for performance profile collector pods to become ready: 2
2026-07-21T07:58:31.799431166Z Gathering data for ns/openshift-image-registry...
2026-07-21T07:58:32.223597777Z Gathering data for ns/openshift-ingress-operator...
2026-07-21T07:58:32.528091949Z Gathering data for ns/openshift-ingress...
2026-07-21T07:58:32.739014512Z Waiting for performance profile collector pods to become ready: 3
2026-07-21T07:58:32.845018191Z Gathering data for ns/openshift-ingress-canary...
2026-07-21T07:58:33.202275460Z Gathering data for ns/openshift-insights...
2026-07-21T07:58:33.850023640Z Waiting for performance profile collector pods to become ready: 4
2026-07-21T07:58:34.833062605Z Gathering data for ns/openshift-jobset-operator...
2026-07-21T07:58:34.967077556Z Waiting for performance profile collector pods to become ready: 5
2026-07-21T07:58:35.505169445Z Gathering data for ns/openshift-config...
2026-07-21T07:58:35.740476209Z Gathering data for ns/openshift-config-managed...
2026-07-21T07:58:36.083477169Z Waiting for performance profile collector pods to become ready: 6
2026-07-21T07:58:36.144831419Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-21T07:58:36.443212439Z Gathering data for ns/openshift-kube-apiserver...
2026-07-21T07:58:36.469576683Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-21T07:58:36.724857179Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-21T07:58:36.991631416Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-21T07:58:37.192664577Z Waiting for performance profile collector pods to become ready: 7
2026-07-21T07:58:37.260669317Z Gathering data for ns/openshift-kube-scheduler...
2026-07-21T07:58:37.501978875Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-21T07:58:37.763098186Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-21T07:58:38.054293529Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-21T07:58:38.313764908Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-21T07:58:38.344776491Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-21T07:58:38.556066933Z Collecting performance related data for node ip-10-0-130-251.ec2.internal
2026-07-21T07:58:38.562887051Z Collecting performance related data for node ip-10-0-137-224.ec2.internal
2026-07-21T07:58:38.573186243Z Collecting performance related data for node ip-10-0-142-172.ec2.internal
2026-07-21T07:58:38.920131187Z Gathering data for ns/openshift-multus...
2026-07-21T07:58:40.390503721Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-21T07:58:41.475653701Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-21T07:58:41.688288222Z Gathering data for ns/openshift-host-network...
2026-07-21T07:58:41.981086351Z Gathering data for ns/openshift-network-diagnostics...
2026-07-21T07:58:42.488322095Z Gathering data for ns/openshift-network-node-identity...
2026-07-21T07:58:42.919465545Z Gathering data for ns/openshift-network-operator...
2026-07-21T07:58:43.283467392Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-21T07:58:43.532797078Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-21T07:58:43.993987070Z Gathering data for ns/openshift-apiserver-operator...
2026-07-21T07:58:44.309574241Z Gathering data for ns/openshift-apiserver...
2026-07-21T07:58:44.677312977Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-21T07:58:44.909623536Z Gathering data for ns/openshift-controller-manager...
2026-07-21T07:58:45.218228339Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-21T07:58:45.872379485Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-21T07:58:46.152062845Z Gathering data for ns/openshift-service-ca-operator...
2026-07-21T07:58:46.241430822Z Collecting kubelet logs for node ip-10-0-130-251.ec2.internal
2026-07-21T07:58:46.243172081Z Collecting kubelet logs for node ip-10-0-137-224.ec2.internal
2026-07-21T07:58:46.244854007Z Collecting kubelet logs for node ip-10-0-142-172.ec2.internal
2026-07-21T07:58:46.480786654Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-21T07:58:46.485081460Z Gathering data for ns/openshift-service-ca...
2026-07-21T07:58:46.584619337Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-kh7c9 namespace
2026-07-21T07:58:46.589275006Z INFO: Node performance data collection complete.
2026-07-21T07:58:46.850596509Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-21T07:58:47.712415943Z Wrote inspect data to must-gather.
2026-07-21T07:58:47.712456681Z error: inspection completed with the errors occurred while gathering data:
2026-07-21T07:58:47.712456681Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-21T07:58:47.830473589Z error: the server doesn't have a resource type "clusters"
2026-07-21T07:58:47.961097618Z Caches written to disk