2026-07-20T19:15:41.380999545Z [disk usage checker] Started
2026-07-20T19:15:41.384516455Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:15:41.584046942Z Gathering data for ns/openshift-cluster-version...
2026-07-20T19:15:41.711250986Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T19:15:41.789960473Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:15:41.863695980Z Gathering data for ns/default...
2026-07-20T19:15:42.151156803Z Gathering data for ns/openshift...
2026-07-20T19:15:42.545330144Z Gathering data for ns/kube-system...
2026-07-20T19:15:43.291297250Z Gathering data for ns/openshift-etcd...
2026-07-20T19:15:43.579991028Z Wrote inspect data to must-gather.
2026-07-20T19:15:43.580074561Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T19:15:43.580074561Z     namespaces "assisted-installer" not found
2026-07-20T19:15:43.921819473Z Waiting on subprocesses to finish execution.
2026-07-20T19:15:43.930752820Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-20T19:15:43.933572031Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-20T19:15:43.935293992Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-20T19:15:43.939110139Z Executing Istio gather script
2026-07-20T19:15:43.963182756Z INFO: Gathering HAProxy config files
2026-07-20T19:15:43.972066444Z INFO: Collecting host service logs for crio
2026-07-20T19:15:43.972415551Z INFO: Collecting host service logs for kubelet
2026-07-20T19:15:43.972751574Z INFO: Collecting host service logs for rpm-ostreed
2026-07-20T19:15:43.973090471Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-20T19:15:43.973451082Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-20T19:15:43.973768539Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-20T19:15:43.974134090Z INFO: Collecting host service logs for NetworkManager
2026-07-20T19:15:43.974470471Z INFO: Collecting host service logs for openvswitch
2026-07-20T19:15:43.974818512Z INFO: Collecting host service logs for ovs-configuration
2026-07-20T19:15:43.975158851Z INFO: Collecting host service logs for ovsdb-server
2026-07-20T19:15:43.975588111Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-20T19:15:43.976076983Z INFO: Waiting for worker host service log collection to complete ...
2026-07-20T19:15:43.994598825Z INFO: Waiting for node performance related collection to complete ...
2026-07-20T19:15:44.398473688Z Wrote inspect data to must-gather.
2026-07-20T19:15:44.577200179Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-20T19:15:44.834915420Z No resources found in openshift-etcd namespace.
2026-07-20T19:15:44.938871678Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-20T19:15:44.938901202Z INFO: on-disk MachineConfig config collection complete.
2026-07-20T19:15:45.042719416Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-20T19:15:45.212709660Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-20T19:15:45.432060967Z error: the server doesn't have a resource type "performanceprofile"
2026-07-20T19:15:45.481728089Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-20T19:15:45.588064018Z No resources found
2026-07-20T19:15:45.635389433Z Wrote inspect data to must-gather.
2026-07-20T19:15:45.637845553Z INFO: "metallb-operator" not detected. Skipping.
2026-07-20T19:15:45.865460624Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-z9b2h
2026-07-20T19:15:45.865507722Z insights-runtime-extractor-4qtjs
2026-07-20T19:15:45.865525283Z insights-runtime-extractor-659m2
2026-07-20T19:15:45.865542980Z insights-runtime-extractor-9fsc2
2026-07-20T19:15:45.915148593Z Gathering data for ns/openshift-monitoring...
2026-07-20T19:15:45.916844898Z Wrote inspect data to must-gather.
2026-07-20T19:15:45.949505591Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-20T19:15:46.122543907Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-20T19:15:46.122543907Z See 'oc rsync -h' for help and examples
2026-07-20T19:15:46.143123324Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:15:46.221214427Z INFO: 'previous-logs' folder not found on ip-10-0-131-91.ec2.internal, skipping...
2026-07-20T19:15:46.234546909Z Wrote inspect data to must-gather.
2026-07-20T19:15:46.252884813Z INFO: Worker host service log collection to complete.
2026-07-20T19:15:46.253004828Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-20T19:15:46.265688330Z No resources found
2026-07-20T19:15:46.387036172Z ERROR: No running kube-apiserver pods found
2026-07-20T19:15:46.425309003Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:15:46.470824806Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-20T19:15:46.604673599Z Wrote inspect data to must-gather.
2026-07-20T19:15:46.608516331Z Wrote inspect data to must-gather.
2026-07-20T19:15:46.615728362Z Wrote inspect data to must-gather.
2026-07-20T19:15:46.618925771Z error: the server doesn't have a resource type "machineconfigs"
2026-07-20T19:15:46.627870051Z Wrote inspect data to must-gather.
2026-07-20T19:15:46.628086196Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-20T19:15:46.710571584Z tar: Removing leading `/' from member names
2026-07-20T19:15:46.717138656Z INFO: HAProxy config collection complete.
2026-07-20T19:15:46.768437543Z INFO: 'previous-logs' folder not found on ip-10-0-133-3.ec2.internal, skipping...
2026-07-20T19:15:46.806711992Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-20T19:15:46.812091545Z Wrote inspect data to must-gather.
2026-07-20T19:15:47.069432155Z INFO: 'previous-logs' folder not found on ip-10-0-139-170.ec2.internal, skipping...
2026-07-20T19:15:47.069544223Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-20T19:15:47.069597510Z INFO: Machine Config Daemon termination log collection complete.
2026-07-20T19:15:47.090055511Z Wrote inspect data to must-gather.
2026-07-20T19:15:47.102071151Z INFO: Getting rules from prometheus-k8s-0
2026-07-20T19:15:47.108776483Z Wrote inspect data to must-gather.
2026-07-20T19:15:47.286733070Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-20T19:15:47.427404810Z INFO: INTERCONNECT MODE
2026-07-20T19:15:47.427444334Z INFO: Gathering ovn-kubernetes DBs
2026-07-20T19:15:47.516585631Z INFO: Getting status/config from prometheus-k8s-0
2026-07-20T19:15:47.532898848Z Wrote inspect data to must-gather.
2026-07-20T19:15:47.607769779Z INFO: Gathering OVN_Northbound from ovnkube-node-bmb9r...
2026-07-20T19:15:47.608122802Z INFO: Gathering OVN_Northbound from ovnkube-node-gbwb2...
2026-07-20T19:15:47.608365871Z INFO: Gathering OVN_Northbound from ovnkube-node-w6d5b...
2026-07-20T19:15:47.608826796Z INFO: Gathering OVN_Southbound from ovnkube-node-bmb9r...
2026-07-20T19:15:47.609119629Z INFO: Gathering OVN_Southbound from ovnkube-node-gbwb2...
2026-07-20T19:15:47.609373462Z INFO: Gathering OVN_Southbound from ovnkube-node-w6d5b...
2026-07-20T19:15:47.708003067Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T19:15:47.924974440Z tar: Removing leading `/' from member names
2026-07-20T19:15:47.945538433Z Wrote inspect data to must-gather.
2026-07-20T19:15:47.957710242Z tar: Removing leading `/' from member names
2026-07-20T19:15:47.968432404Z tar: Removing leading `/' from member names
2026-07-20T19:15:47.970982143Z tar: Removing leading `/' from member names
2026-07-20T19:15:47.987161467Z tar: Removing leading `/' from member names
2026-07-20T19:15:47.996357492Z tar: Removing leading `/' from member names
2026-07-20T19:15:48.018314559Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-20T19:15:48.092304634Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-20T19:15:48.204108013Z Wrote inspect data to must-gather.
2026-07-20T19:15:48.239920543Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:48.383126661Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-20T19:15:48.454040582Z W0720 19:15:48.453998    1392 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-20T19:15:48.459500338Z Wrote inspect data to must-gather.
2026-07-20T19:15:48.486816107Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T19:15:48.494574763Z INFO: Waiting for network log collection to complete ...
2026-07-20T19:15:48.494685300Z INFO: Waiting for ovnk database copies to complete ...
2026-07-20T19:15:48.495222119Z INFO: Copying ovnk databases complete.
2026-07-20T19:15:48.497437339Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-20T19:15:48.501382385Z ovnk_database_store/
2026-07-20T19:15:48.501430018Z ovnk_database_store/ovnkube-node-bmb9r_nbdb
2026-07-20T19:15:48.506276434Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:48.509234634Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-20T19:15:48.510092690Z ovnk_database_store/ovnkube-node-w6d5b_nbdb
2026-07-20T19:15:48.520053278Z ovnk_database_store/ovnkube-node-bmb9r_sbdb
2026-07-20T19:15:48.539055492Z ovnk_database_store/ovnkube-node-gbwb2_nbdb
2026-07-20T19:15:48.545897477Z ovnk_database_store/ovnkube-node-gbwb2_sbdb
2026-07-20T19:15:48.563844370Z ovnk_database_store/ovnkube-node-w6d5b_sbdb
2026-07-20T19:15:48.584497575Z INFO: Network log collection complete.
2026-07-20T19:15:48.659941190Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-20T19:15:48.688869248Z Gathering data for ns/openshift-network-console...
2026-07-20T19:15:48.724139155Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:48.779051541Z Wrote inspect data to must-gather.
2026-07-20T19:15:48.921306234Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-20T19:15:48.943069487Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:48.946138152Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-20T19:15:49.077510183Z Wrote inspect data to must-gather.
2026-07-20T19:15:49.161161297Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:49.184163847Z Gathering data for ns/openshift-console-operator...
2026-07-20T19:15:49.335968130Z INFO: Getting status from alertmanager-main-0
2026-07-20T19:15:49.390147622Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:49.392343096Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-20T19:15:49.395397625Z Wrote inspect data to must-gather.
2026-07-20T19:15:49.648817356Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:49.654764496Z Gathering data for ns/openshift-console...
2026-07-20T19:15:49.660096496Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-20T19:15:49.662015882Z INFO: Fallback to identify the container image from release info
2026-07-20T19:15:49.847955922Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:49.850170264Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-20T19:15:50.004537538Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:50.197321189Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-20T19:15:50.213309750Z Wrote inspect data to /must-gather/istio.
2026-07-20T19:15:50.224073728Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-20T19:15:50.329869861Z daemonset.apps/perf-node-gather-daemonset created
2026-07-20T19:15:50.445055923Z Waiting for performance profile collector pods to become ready: 1
2026-07-20T19:15:50.908289853Z Gathering data for ns/openshift-dns-operator...
2026-07-20T19:15:50.910356815Z Done executing Istio gather script
2026-07-20T19:15:51.258038790Z Gathering data for ns/openshift-dns...
2026-07-20T19:15:51.449045029Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:15:51.565854462Z Waiting for performance profile collector pods to become ready: 2
2026-07-20T19:15:51.852450691Z Gathering data for ns/openshift-image-registry...
2026-07-20T19:15:52.315317314Z Gathering data for ns/openshift-ingress-operator...
2026-07-20T19:15:52.679075529Z Waiting for performance profile collector pods to become ready: 3
2026-07-20T19:15:52.680669515Z Gathering data for ns/openshift-ingress...
2026-07-20T19:15:53.030068286Z Gathering data for ns/openshift-ingress-canary...
2026-07-20T19:15:53.435921937Z Gathering data for ns/openshift-insights...
2026-07-20T19:15:53.790866350Z Waiting for performance profile collector pods to become ready: 4
2026-07-20T19:15:54.900703660Z Waiting for performance profile collector pods to become ready: 5
2026-07-20T19:15:55.332661190Z Gathering data for ns/openshift-jobset-operator...
2026-07-20T19:15:56.008737679Z Waiting for performance profile collector pods to become ready: 6
2026-07-20T19:15:56.157621197Z Gathering data for ns/openshift-config...
2026-07-20T19:15:56.454277013Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:15:56.494647027Z Gathering data for ns/openshift-config-managed...
2026-07-20T19:15:56.928100184Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-20T19:15:57.106308239Z Waiting for performance profile collector pods to become ready: 7
2026-07-20T19:15:57.309257164Z Gathering data for ns/openshift-kube-apiserver...
2026-07-20T19:15:57.591331541Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-20T19:15:57.907633768Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-20T19:15:58.204709185Z Gathering data for ns/openshift-kube-scheduler...
2026-07-20T19:15:58.224667606Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-20T19:15:58.475491213Z Collecting performance related data for node ip-10-0-139-170.ec2.internal
2026-07-20T19:15:58.483184003Z Collecting performance related data for node ip-10-0-131-91.ec2.internal
2026-07-20T19:15:58.491341545Z Collecting performance related data for node ip-10-0-133-3.ec2.internal
2026-07-20T19:15:58.515934640Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-20T19:15:58.932566323Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-20T19:15:59.324461327Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-20T19:15:59.672992121Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-20T19:16:00.257464601Z Gathering data for ns/openshift-multus...
2026-07-20T19:16:01.458844890Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:16:01.751955338Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-20T19:16:02.989691672Z Gathering data for ns/openshift-host-network...
2026-07-20T19:16:03.296391469Z Gathering data for ns/openshift-network-diagnostics...
2026-07-20T19:16:03.852976277Z Gathering data for ns/openshift-network-node-identity...
2026-07-20T19:16:04.302687175Z Gathering data for ns/openshift-network-operator...
2026-07-20T19:16:04.692222160Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-20T19:16:05.014073473Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-20T19:16:05.526846475Z Gathering data for ns/openshift-apiserver-operator...
2026-07-20T19:16:05.825734405Z Gathering data for ns/openshift-apiserver...
2026-07-20T19:16:06.209971227Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-20T19:16:06.464309411Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-20T19:16:06.465216660Z Collecting kubelet logs for node ip-10-0-139-170.ec2.internal
2026-07-20T19:16:06.466861996Z Collecting kubelet logs for node ip-10-0-131-91.ec2.internal
2026-07-20T19:16:06.468488659Z Collecting kubelet logs for node ip-10-0-133-3.ec2.internal
2026-07-20T19:16:06.474892490Z Gathering data for ns/openshift-controller-manager...
2026-07-20T19:16:06.798602170Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-20T19:16:06.811957602Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-pxtbp namespace
2026-07-20T19:16:06.816695314Z INFO: Node performance data collection complete.
2026-07-20T19:16:07.467874486Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-20T19:16:07.750552579Z Gathering data for ns/openshift-service-ca-operator...
2026-07-20T19:16:08.050437145Z Gathering data for ns/openshift-service-ca...
2026-07-20T19:16:08.412106103Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-20T19:16:09.317862057Z Wrote inspect data to must-gather.
2026-07-20T19:16:09.317907375Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T19:16:09.317907375Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-20T19:16:09.442313492Z error: the server doesn't have a resource type "clusters"
2026-07-20T19:16:09.579821849Z Caches written to disk