2026-07-24T18:16:46.427484125Z [disk usage checker] Started
2026-07-24T18:16:46.430683659Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T18:16:46.648096134Z Gathering data for ns/openshift-cluster-version...
2026-07-24T18:16:46.800628993Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T18:16:46.902656404Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T18:16:46.992846001Z Gathering data for ns/default...
2026-07-24T18:16:47.334623415Z Gathering data for ns/openshift...
2026-07-24T18:16:47.786453692Z Gathering data for ns/kube-system...
2026-07-24T18:16:48.578148182Z Gathering data for ns/openshift-etcd...
2026-07-24T18:16:48.950866970Z Wrote inspect data to must-gather.
2026-07-24T18:16:48.950911592Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T18:16:48.950911592Z     namespaces "assisted-installer" not found
2026-07-24T18:16:49.267381613Z Waiting on subprocesses to finish execution.
2026-07-24T18:16:49.274323016Z INFO: Gathering HAProxy config files
2026-07-24T18:16:49.282398630Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-24T18:16:49.284461367Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-24T18:16:49.293303959Z Executing Istio gather script
2026-07-24T18:16:49.306657014Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-24T18:16:49.329203103Z INFO: Collecting host service logs for crio
2026-07-24T18:16:49.329767324Z INFO: Collecting host service logs for kubelet
2026-07-24T18:16:49.330137386Z INFO: Collecting host service logs for rpm-ostreed
2026-07-24T18:16:49.330506887Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-24T18:16:49.330864015Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-24T18:16:49.331230053Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-24T18:16:49.331602829Z INFO: Collecting host service logs for NetworkManager
2026-07-24T18:16:49.331952385Z INFO: Collecting host service logs for openvswitch
2026-07-24T18:16:49.332276524Z INFO: Collecting host service logs for ovs-configuration
2026-07-24T18:16:49.332616737Z INFO: Collecting host service logs for ovsdb-server
2026-07-24T18:16:49.332960423Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-24T18:16:49.333347120Z INFO: Waiting for worker host service log collection to complete ...
2026-07-24T18:16:49.352329122Z INFO: Waiting for node performance related collection to complete ...
2026-07-24T18:16:49.571031798Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-24T18:16:49.866246318Z No resources found
2026-07-24T18:16:49.929728127Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-24T18:16:50.185559581Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-24T18:16:50.185640890Z INFO: on-disk MachineConfig config collection complete.
2026-07-24T18:16:50.315653496Z Wrote inspect data to must-gather.
2026-07-24T18:16:50.518057246Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-24T18:16:50.573364336Z INFO: "metallb-operator" not detected. Skipping.
2026-07-24T18:16:50.643218383Z Wrote inspect data to must-gather.
2026-07-24T18:16:50.660697886Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-24T18:16:50.703228309Z error: the server doesn't have a resource type "performanceprofile"
2026-07-24T18:16:50.722986575Z No resources found in openshift-etcd namespace.
2026-07-24T18:16:50.874358202Z ERROR: No running kube-apiserver pods found
2026-07-24T18:16:50.940429579Z INFO: Worker host service log collection to complete.
2026-07-24T18:16:50.964048996Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-24T18:16:50.989588835Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-4wr89
2026-07-24T18:16:50.989588835Z insights-runtime-extractor-dpwdf
2026-07-24T18:16:50.989588835Z insights-runtime-extractor-s9qpk
2026-07-24T18:16:50.989588835Z insights-runtime-extractor-xd6x2
2026-07-24T18:16:51.215276177Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-24T18:16:51.562538418Z Wrote inspect data to must-gather.
2026-07-24T18:16:51.655793698Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T18:16:51.700717847Z INFO: 'previous-logs' folder not found on ip-10-0-128-27.ec2.internal, skipping...
2026-07-24T18:16:51.731185741Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T18:16:51.748320997Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-24T18:16:51.788334841Z Wrote inspect data to must-gather.
2026-07-24T18:16:51.824399087Z No resources found
2026-07-24T18:16:51.881261277Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-24T18:16:51.881261277Z See 'oc rsync -h' for help and examples
2026-07-24T18:16:51.909320426Z Wrote inspect data to must-gather.
2026-07-24T18:16:51.960040610Z Gathering data for ns/openshift-monitoring...
2026-07-24T18:16:51.961791377Z tar: Removing leading `/' from member names
2026-07-24T18:16:51.972392024Z INFO: HAProxy config collection complete.
2026-07-24T18:16:51.993106958Z Wrote inspect data to must-gather.
2026-07-24T18:16:52.007916351Z Wrote inspect data to must-gather.
2026-07-24T18:16:52.127632574Z error: the server doesn't have a resource type "machineconfigs"
2026-07-24T18:16:52.187968211Z INFO: 'previous-logs' folder not found on ip-10-0-139-5.ec2.internal, skipping...
2026-07-24T18:16:52.305953052Z INFO: Getting rules from prometheus-k8s-0
2026-07-24T18:16:52.313421408Z Wrote inspect data to must-gather.
2026-07-24T18:16:52.335906884Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-24T18:16:52.497282512Z INFO: 'previous-logs' folder not found on ip-10-0-140-128.ec2.internal, skipping...
2026-07-24T18:16:52.497406557Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-24T18:16:52.497469190Z INFO: Machine Config Daemon termination log collection complete.
2026-07-24T18:16:52.627964070Z Wrote inspect data to must-gather.
2026-07-24T18:16:52.628055933Z Wrote inspect data to must-gather.
2026-07-24T18:16:52.698197109Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-24T18:16:52.810484664Z INFO: Getting status/config from prometheus-k8s-0
2026-07-24T18:16:52.817845909Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-24T18:16:52.928111359Z Wrote inspect data to must-gather.
2026-07-24T18:16:53.045304655Z Wrote inspect data to must-gather.
2026-07-24T18:16:53.246295413Z INFO: INTERCONNECT MODE
2026-07-24T18:16:53.246382832Z INFO: Gathering ovn-kubernetes DBs
2026-07-24T18:16:53.251730223Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-24T18:16:53.268314976Z Wrote inspect data to must-gather.
2026-07-24T18:16:53.424118414Z INFO: Gathering OVN_Northbound from ovnkube-node-55x6j...
2026-07-24T18:16:53.424444116Z INFO: Gathering OVN_Northbound from ovnkube-node-sfrqx...
2026-07-24T18:16:53.424699955Z INFO: Gathering OVN_Northbound from ovnkube-node-w5tt2...
2026-07-24T18:16:53.425007613Z INFO: Gathering OVN_Southbound from ovnkube-node-55x6j...
2026-07-24T18:16:53.425258586Z INFO: Gathering OVN_Southbound from ovnkube-node-sfrqx...
2026-07-24T18:16:53.425602664Z INFO: Gathering OVN_Southbound from ovnkube-node-w5tt2...
2026-07-24T18:16:53.554445231Z Wrote inspect data to must-gather.
2026-07-24T18:16:53.624046102Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-24T18:16:53.779168030Z tar: Removing leading `/' from member names
2026-07-24T18:16:53.784231020Z tar: Removing leading `/' from member names
2026-07-24T18:16:53.792105987Z tar: Removing leading `/' from member names
2026-07-24T18:16:53.796233450Z tar: Removing leading `/' from member names
2026-07-24T18:16:53.799154151Z tar: Removing leading `/' from member names
2026-07-24T18:16:53.804949588Z tar: Removing leading `/' from member names
2026-07-24T18:16:53.912845450Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T18:16:53.921382034Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-24T18:16:53.937549255Z Wrote inspect data to must-gather.
2026-07-24T18:16:53.972168326Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-24T18:16:54.179556064Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:54.209954275Z Wrote inspect data to must-gather.
2026-07-24T18:16:54.276285293Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-24T18:16:54.352710554Z INFO: Waiting for network log collection to complete ...
2026-07-24T18:16:54.352745190Z INFO: Waiting for ovnk database copies to complete ...
2026-07-24T18:16:54.353272626Z INFO: Copying ovnk databases complete.
2026-07-24T18:16:54.355506342Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-24T18:16:54.361685667Z ovnk_database_store/
2026-07-24T18:16:54.361737798Z ovnk_database_store/ovnkube-node-sfrqx_sbdb
2026-07-24T18:16:54.391263742Z ovnk_database_store/ovnkube-node-w5tt2_nbdb
2026-07-24T18:16:54.398343208Z ovnk_database_store/ovnkube-node-sfrqx_nbdb
2026-07-24T18:16:54.407442507Z ovnk_database_store/ovnkube-node-55x6j_nbdb
2026-07-24T18:16:54.413531290Z ovnk_database_store/ovnkube-node-55x6j_sbdb
2026-07-24T18:16:54.429487822Z ovnk_database_store/ovnkube-node-w5tt2_sbdb
2026-07-24T18:16:54.437098576Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:54.443041103Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-24T18:16:54.450953133Z INFO: Network log collection complete.
2026-07-24T18:16:54.469191131Z W0724 18:16:54.469143    1491 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-24T18:16:54.476419281Z Wrote inspect data to must-gather.
2026-07-24T18:16:54.563000898Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T18:16:54.683831122Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:54.733880890Z INFO: Getting status from alertmanager-main-0
2026-07-24T18:16:54.783440743Z Wrote inspect data to must-gather.
2026-07-24T18:16:54.784916084Z Gathering data for ns/openshift-network-console...
2026-07-24T18:16:54.930171407Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:54.933864482Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-24T18:16:55.078949896Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-24T18:16:55.081633459Z INFO: Fallback to identify the container image from release info
2026-07-24T18:16:55.135435687Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:55.334656200Z Gathering data for ns/openshift-console-operator...
2026-07-24T18:16:55.358939091Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:55.361789140Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-24T18:16:55.593539472Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:55.673276000Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-24T18:16:55.814451308Z Gathering data for ns/openshift-console...
2026-07-24T18:16:55.816933016Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:55.819617109Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-24T18:16:55.847015543Z daemonset.apps/perf-node-gather-daemonset created
2026-07-24T18:16:55.988034248Z Waiting for performance profile collector pods to become ready: 1
2026-07-24T18:16:56.008788527Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:56.196941584Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:56.199488074Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-24T18:16:56.414704178Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:56.480929884Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-24T18:16:56.636407859Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:56.639421924Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-24T18:16:56.749803934Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T18:16:56.812324986Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:57.034161772Z Wrote inspect data to /must-gather/istio.
2026-07-24T18:16:57.070722711Z Gathering data for ns/openshift-dns-operator...
2026-07-24T18:16:57.156650377Z Waiting for performance profile collector pods to become ready: 2
2026-07-24T18:16:57.440473143Z Gathering data for ns/openshift-dns...
2026-07-24T18:16:57.730700522Z Done executing Istio gather script
2026-07-24T18:16:58.181132927Z Gathering data for ns/openshift-image-registry...
2026-07-24T18:16:58.278839899Z Waiting for performance profile collector pods to become ready: 3
2026-07-24T18:16:58.680045661Z Gathering data for ns/openshift-ingress-operator...
2026-07-24T18:16:59.061156669Z Gathering data for ns/openshift-ingress...
2026-07-24T18:16:59.426144434Z Waiting for performance profile collector pods to become ready: 4
2026-07-24T18:16:59.453134295Z Gathering data for ns/openshift-ingress-canary...
2026-07-24T18:16:59.869226280Z Gathering data for ns/openshift-insights...
2026-07-24T18:17:00.553521282Z Waiting for performance profile collector pods to become ready: 5
2026-07-24T18:17:01.684785235Z Waiting for performance profile collector pods to become ready: 6
2026-07-24T18:17:01.754375825Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T18:17:02.500882490Z Gathering data for ns/kserve...
2026-07-24T18:17:02.804335547Z Waiting for performance profile collector pods to become ready: 7
2026-07-24T18:17:03.861073925Z Gathering data for ns/openshift-config...
2026-07-24T18:17:03.949753305Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-24T18:17:04.211919100Z Gathering data for ns/openshift-config-managed...
2026-07-24T18:17:04.217837169Z Collecting performance related data for node ip-10-0-128-27.ec2.internal
2026-07-24T18:17:04.228618515Z Collecting performance related data for node ip-10-0-140-128.ec2.internal
2026-07-24T18:17:04.246277739Z Collecting performance related data for node ip-10-0-139-5.ec2.internal
2026-07-24T18:17:04.742519852Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-24T18:17:05.127381255Z Gathering data for ns/openshift-kube-apiserver...
2026-07-24T18:17:05.502055631Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-24T18:17:05.915654533Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-24T18:17:06.240826105Z Gathering data for ns/openshift-kube-scheduler...
2026-07-24T18:17:06.600916891Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-24T18:17:06.759958317Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T18:17:06.996341381Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-24T18:17:07.412906797Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-24T18:17:07.788960298Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-24T18:17:08.388682755Z Gathering data for ns/openshift-multus...
2026-07-24T18:17:10.129109032Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-24T18:17:11.567353154Z Gathering data for ns/openshift-host-network...
2026-07-24T18:17:11.765356745Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T18:17:11.927026794Z Gathering data for ns/openshift-network-diagnostics...
2026-07-24T18:17:12.535367008Z Gathering data for ns/openshift-network-node-identity...
2026-07-24T18:17:12.643020223Z Collecting kubelet logs for node ip-10-0-128-27.ec2.internal
2026-07-24T18:17:12.644689221Z Collecting kubelet logs for node ip-10-0-140-128.ec2.internal
2026-07-24T18:17:12.646810280Z Collecting kubelet logs for node ip-10-0-139-5.ec2.internal
2026-07-24T18:17:13.011903971Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-btkjn namespace
2026-07-24T18:17:13.019075544Z INFO: Node performance data collection complete.
2026-07-24T18:17:13.055229345Z Gathering data for ns/openshift-network-operator...
2026-07-24T18:17:13.488552340Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-24T18:17:13.781433802Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-24T18:17:14.310089593Z Gathering data for ns/openshift-apiserver-operator...
2026-07-24T18:17:14.634948174Z Gathering data for ns/openshift-apiserver...
2026-07-24T18:17:15.114401604Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-24T18:17:15.403223735Z Gathering data for ns/openshift-controller-manager...
2026-07-24T18:17:15.715489528Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-24T18:17:16.423131278Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-24T18:17:16.736061005Z Gathering data for ns/openshift-service-ca-operator...
2026-07-24T18:17:16.770573667Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T18:17:17.108546742Z Gathering data for ns/openshift-service-ca...
2026-07-24T18:17:17.543124529Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-24T18:17:18.485799063Z Wrote inspect data to must-gather.
2026-07-24T18:17:18.485957892Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T18:17:18.485957892Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-24T18:17:18.620074725Z error: the server doesn't have a resource type "clusters"
2026-07-24T18:17:18.756270976Z Caches written to disk