2026-07-06T20:48:24.656755700Z [disk usage checker] Started
2026-07-06T20:48:24.659968592Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T20:48:24.852003148Z Gathering data for ns/openshift-cluster-version...
2026-07-06T20:48:24.991132716Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T20:48:25.087237320Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T20:48:25.178348430Z Gathering data for ns/default...
2026-07-06T20:48:25.502686774Z Gathering data for ns/openshift...
2026-07-06T20:48:25.921891986Z Gathering data for ns/kube-system...
2026-07-06T20:48:26.701272244Z Gathering data for ns/openshift-etcd...
2026-07-06T20:48:27.072973836Z Wrote inspect data to must-gather.
2026-07-06T20:48:27.073013980Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T20:48:27.073013980Z     namespaces "assisted-installer" not found
2026-07-06T20:48:27.169114952Z Waiting on subprocesses to finish execution.
2026-07-06T20:48:27.175356876Z INFO: Gathering HAProxy config files
2026-07-06T20:48:27.178467774Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-06T20:48:27.181276574Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-06T20:48:27.181656709Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-06T20:48:27.191353711Z Executing Istio gather script
2026-07-06T20:48:27.215685249Z INFO: Collecting host service logs for crio
2026-07-06T20:48:27.215962185Z INFO: Collecting host service logs for kubelet
2026-07-06T20:48:27.216292789Z INFO: Collecting host service logs for rpm-ostreed
2026-07-06T20:48:27.216598681Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-06T20:48:27.218010096Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-06T20:48:27.218362792Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-06T20:48:27.218704623Z INFO: Collecting host service logs for NetworkManager
2026-07-06T20:48:27.219083832Z INFO: Collecting host service logs for openvswitch
2026-07-06T20:48:27.219427963Z INFO: Collecting host service logs for ovs-configuration
2026-07-06T20:48:27.219761165Z INFO: Collecting host service logs for ovsdb-server
2026-07-06T20:48:27.220121892Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-06T20:48:27.220648593Z INFO: Waiting for worker host service log collection to complete ...
2026-07-06T20:48:27.262671365Z INFO: Waiting for node performance related collection to complete ...
2026-07-06T20:48:27.624939665Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-06T20:48:27.887460873Z No resources found
2026-07-06T20:48:27.964193596Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-zqssl
2026-07-06T20:48:27.964313338Z insights-runtime-extractor-8s8rq
2026-07-06T20:48:27.964338052Z insights-runtime-extractor-f26zg
2026-07-06T20:48:27.964358656Z insights-runtime-extractor-qhpqg
2026-07-06T20:48:28.261138735Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-06T20:48:28.261236298Z INFO: on-disk MachineConfig config collection complete.
2026-07-06T20:48:28.307118704Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-06T20:48:28.307598457Z Wrote inspect data to must-gather.
2026-07-06T20:48:28.350220939Z No resources found in openshift-etcd namespace.
2026-07-06T20:48:28.362285817Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-06T20:48:28.433124748Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-06T20:48:28.433124748Z See 'oc rsync -h' for help and examples
2026-07-06T20:48:28.463260008Z INFO: "metallb-operator" not detected. Skipping.
2026-07-06T20:48:28.524030050Z error: the server doesn't have a resource type "performanceprofile"
2026-07-06T20:48:28.556756175Z INFO: Worker host service log collection to complete.
2026-07-06T20:48:28.595165278Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-06T20:48:28.599851620Z ERROR: No running kube-apiserver pods found
2026-07-06T20:48:28.669350150Z Wrote inspect data to must-gather.
2026-07-06T20:48:28.924222850Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-06T20:48:28.935923130Z INFO: 'previous-logs' folder not found on ip-10-0-128-198.ec2.internal, skipping...
2026-07-06T20:48:28.998869875Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T20:48:29.276532357Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-06T20:48:29.306469841Z Gathering data for ns/openshift-monitoring...
2026-07-06T20:48:29.397171973Z Wrote inspect data to must-gather.
2026-07-06T20:48:29.508005998Z No resources found
2026-07-06T20:48:29.515680426Z Wrote inspect data to must-gather.
2026-07-06T20:48:29.527962685Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-06T20:48:29.629185606Z Wrote inspect data to must-gather.
2026-07-06T20:48:29.638284022Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-06T20:48:29.661392321Z Wrote inspect data to must-gather.
2026-07-06T20:48:29.669360649Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T20:48:29.683817932Z INFO: 'previous-logs' folder not found on ip-10-0-133-209.ec2.internal, skipping...
2026-07-06T20:48:29.693667912Z Wrote inspect data to must-gather.
2026-07-06T20:48:29.766645896Z error: the server doesn't have a resource type "machineconfigs"
2026-07-06T20:48:29.797801078Z tar: Removing leading `/' from member names
2026-07-06T20:48:29.803306509Z INFO: HAProxy config collection complete.
2026-07-06T20:48:29.930036264Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-06T20:48:29.944547297Z Wrote inspect data to must-gather.
2026-07-06T20:48:29.947621326Z INFO: 'previous-logs' folder not found on ip-10-0-134-228.ec2.internal, skipping...
2026-07-06T20:48:29.947720238Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-06T20:48:29.948206453Z INFO: Machine Config Daemon termination log collection complete.
2026-07-06T20:48:30.041506067Z INFO: Getting rules from prometheus-k8s-0
2026-07-06T20:48:30.139762303Z Wrote inspect data to must-gather.
2026-07-06T20:48:30.199610290Z Wrote inspect data to must-gather.
2026-07-06T20:48:30.302713629Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-06T20:48:30.439054517Z Wrote inspect data to must-gather.
2026-07-06T20:48:30.451579794Z INFO: Getting status/config from prometheus-k8s-0
2026-07-06T20:48:30.519957987Z Wrote inspect data to must-gather.
2026-07-06T20:48:30.575505729Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-06T20:48:30.701093484Z INFO: INTERCONNECT MODE
2026-07-06T20:48:30.701122687Z INFO: Gathering ovn-kubernetes DBs
2026-07-06T20:48:30.722287409Z Wrote inspect data to must-gather.
2026-07-06T20:48:30.747901933Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:30.860260626Z INFO: Gathering OVN_Northbound from ovnkube-node-8mbnx...
2026-07-06T20:48:30.860532880Z INFO: Gathering OVN_Northbound from ovnkube-node-jx5zq...
2026-07-06T20:48:30.860731089Z INFO: Gathering OVN_Northbound from ovnkube-node-ltvbf...
2026-07-06T20:48:30.861004238Z INFO: Gathering OVN_Southbound from ovnkube-node-8mbnx...
2026-07-06T20:48:30.861391990Z INFO: Gathering OVN_Southbound from ovnkube-node-jx5zq...
2026-07-06T20:48:30.861865651Z INFO: Gathering OVN_Southbound from ovnkube-node-ltvbf...
2026-07-06T20:48:30.887330217Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-06T20:48:31.037520686Z Wrote inspect data to must-gather.
2026-07-06T20:48:31.068201177Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:31.075204643Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-06T20:48:31.189950284Z tar: Removing leading `/' from member names
2026-07-06T20:48:31.233161342Z tar: Removing leading `/' from member names
2026-07-06T20:48:31.244676728Z tar: Removing leading `/' from member names
2026-07-06T20:48:31.247655237Z tar: Removing leading `/' from member names
2026-07-06T20:48:31.253519835Z tar: Removing leading `/' from member names
2026-07-06T20:48:31.264904601Z tar: Removing leading `/' from member names
2026-07-06T20:48:31.309486312Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-06T20:48:31.328243307Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T20:48:31.352095311Z Wrote inspect data to must-gather.
2026-07-06T20:48:31.371156101Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:31.574436752Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-06T20:48:31.607920727Z Wrote inspect data to must-gather.
2026-07-06T20:48:31.638226213Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:31.641389456Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-06T20:48:31.707907595Z INFO: Waiting for network log collection to complete ...
2026-07-06T20:48:31.707945639Z INFO: Waiting for ovnk database copies to complete ...
2026-07-06T20:48:31.708318644Z INFO: Copying ovnk databases complete.
2026-07-06T20:48:31.709897566Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-06T20:48:31.712288630Z ovnk_database_store/
2026-07-06T20:48:31.712288630Z ovnk_database_store/ovnkube-node-ltvbf_sbdb
2026-07-06T20:48:31.729882448Z ovnk_database_store/ovnkube-node-ltvbf_nbdb
2026-07-06T20:48:31.737963931Z ovnk_database_store/ovnkube-node-8mbnx_nbdb
2026-07-06T20:48:31.743826775Z ovnk_database_store/ovnkube-node-jx5zq_nbdb
2026-07-06T20:48:31.749685210Z ovnk_database_store/ovnkube-node-8mbnx_sbdb
2026-07-06T20:48:31.765928166Z ovnk_database_store/ovnkube-node-jx5zq_sbdb
2026-07-06T20:48:31.785347253Z INFO: Network log collection complete.
2026-07-06T20:48:31.831936870Z Wrote inspect data to must-gather.
2026-07-06T20:48:31.845100055Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:31.860928709Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-06T20:48:31.998326905Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T20:48:32.071654692Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:32.074430058Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-06T20:48:32.131502304Z Wrote inspect data to must-gather.
2026-07-06T20:48:32.210098048Z Gathering data for ns/openshift-network-console...
2026-07-06T20:48:32.295166604Z INFO: Getting status from alertmanager-main-0
2026-07-06T20:48:32.303936239Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:32.429050603Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-06T20:48:32.431093663Z INFO: Fallback to identify the container image from release info
2026-07-06T20:48:32.504745871Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:32.509892017Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-06T20:48:32.666018804Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:32.782428390Z Gathering data for ns/openshift-console-operator...
2026-07-06T20:48:32.849977651Z Wrote inspect data to /must-gather/istio.
2026-07-06T20:48:33.034672066Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-06T20:48:33.170044262Z daemonset.apps/perf-node-gather-daemonset created
2026-07-06T20:48:33.303351482Z Gathering data for ns/openshift-console...
2026-07-06T20:48:33.304176924Z Waiting for performance profile collector pods to become ready: 1
2026-07-06T20:48:33.550704895Z Done executing Istio gather script
2026-07-06T20:48:33.969938157Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-06T20:48:34.421279105Z Waiting for performance profile collector pods to become ready: 2
2026-07-06T20:48:34.564968937Z Gathering data for ns/openshift-dns-operator...
2026-07-06T20:48:34.675019808Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T20:48:34.930449825Z Gathering data for ns/openshift-dns...
2026-07-06T20:48:35.543083256Z Waiting for performance profile collector pods to become ready: 3
2026-07-06T20:48:35.643901701Z Gathering data for ns/openshift-image-registry...
2026-07-06T20:48:36.169618247Z Gathering data for ns/openshift-ingress-operator...
2026-07-06T20:48:36.530777168Z Gathering data for ns/openshift-ingress...
2026-07-06T20:48:36.663868796Z Waiting for performance profile collector pods to become ready: 4
2026-07-06T20:48:36.905488210Z Gathering data for ns/openshift-ingress-canary...
2026-07-06T20:48:37.342768187Z Gathering data for ns/openshift-insights...
2026-07-06T20:48:37.772326690Z Waiting for performance profile collector pods to become ready: 5
2026-07-06T20:48:38.876493731Z Waiting for performance profile collector pods to become ready: 6
2026-07-06T20:48:39.533663016Z Gathering data for ns/openshift-config...
2026-07-06T20:48:39.679616226Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T20:48:39.837613695Z Gathering data for ns/openshift-config-managed...
2026-07-06T20:48:39.985749463Z Waiting for performance profile collector pods to become ready: 7
2026-07-06T20:48:40.399696122Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-06T20:48:40.862529391Z Gathering data for ns/openshift-kube-apiserver...
2026-07-06T20:48:41.093125137Z Waiting for performance profile collector pods to become ready: 8
2026-07-06T20:48:41.202030665Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-06T20:48:41.546502537Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-06T20:48:41.924914799Z Gathering data for ns/openshift-kube-scheduler...
2026-07-06T20:48:42.213605649Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-06T20:48:42.235982028Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-06T20:48:42.468080910Z Collecting performance related data for node ip-10-0-134-228.ec2.internal
2026-07-06T20:48:42.476604064Z Collecting performance related data for node ip-10-0-128-198.ec2.internal
2026-07-06T20:48:42.484807065Z Collecting performance related data for node ip-10-0-133-209.ec2.internal
2026-07-06T20:48:42.638209238Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-06T20:48:43.055681786Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-06T20:48:43.469230735Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-06T20:48:44.126308891Z Gathering data for ns/openshift-multus...
2026-07-06T20:48:44.684396704Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T20:48:45.915756594Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-06T20:48:47.731385652Z Gathering data for ns/openshift-host-network...
2026-07-06T20:48:48.255690217Z Gathering data for ns/openshift-network-diagnostics...
2026-07-06T20:48:48.848207400Z Gathering data for ns/openshift-network-node-identity...
2026-07-06T20:48:49.278972876Z Gathering data for ns/openshift-network-operator...
2026-07-06T20:48:49.688947256Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T20:48:49.733286448Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-06T20:48:50.078266502Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-06T20:48:50.415725450Z Collecting kubelet logs for node ip-10-0-134-228.ec2.internal
2026-07-06T20:48:50.417920829Z Collecting kubelet logs for node ip-10-0-128-198.ec2.internal
2026-07-06T20:48:50.420025647Z Collecting kubelet logs for node ip-10-0-133-209.ec2.internal
2026-07-06T20:48:50.689631259Z Gathering data for ns/openshift-apiserver-operator...
2026-07-06T20:48:50.762118284Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-2mjld namespace
2026-07-06T20:48:50.767402856Z INFO: Node performance data collection complete.
2026-07-06T20:48:51.066934552Z Gathering data for ns/openshift-apiserver...
2026-07-06T20:48:51.467501485Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-06T20:48:51.786578151Z Gathering data for ns/openshift-controller-manager...
2026-07-06T20:48:52.092383477Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-06T20:48:52.776959751Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-06T20:48:53.138254559Z Gathering data for ns/openshift-service-ca-operator...
2026-07-06T20:48:53.479749553Z Gathering data for ns/openshift-service-ca...
2026-07-06T20:48:53.857197145Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-06T20:48:54.693720324Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T20:48:54.786682859Z Wrote inspect data to must-gather.
2026-07-06T20:48:54.786720254Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T20:48:54.786720254Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-06T20:48:54.906542881Z error: the server doesn't have a resource type "clusters"
2026-07-06T20:48:55.044966369Z Caches written to disk