2026-07-24T11:32:27.809698879Z [disk usage checker] Started
2026-07-24T11:32:27.813036879Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T11:32:28.013948754Z Gathering data for ns/openshift-cluster-version...
2026-07-24T11:32:28.157628523Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T11:32:28.250299043Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T11:32:28.336015656Z Gathering data for ns/default...
2026-07-24T11:32:28.646774684Z Gathering data for ns/openshift...
2026-07-24T11:32:29.107292605Z Gathering data for ns/kube-system...
2026-07-24T11:32:29.821421828Z Gathering data for ns/openshift-etcd...
2026-07-24T11:32:30.148269152Z Wrote inspect data to must-gather.
2026-07-24T11:32:30.148358153Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T11:32:30.148358153Z     namespaces "assisted-installer" not found
2026-07-24T11:32:30.386872110Z Waiting on subprocesses to finish execution.
2026-07-24T11:32:30.395558939Z Executing Istio gather script
2026-07-24T11:32:30.402692478Z INFO: Collecting host service logs for crio
2026-07-24T11:32:30.403027905Z INFO: Collecting host service logs for kubelet
2026-07-24T11:32:30.403578074Z INFO: Collecting host service logs for rpm-ostreed
2026-07-24T11:32:30.403922414Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-24T11:32:30.404303180Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-24T11:32:30.404657519Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-24T11:32:30.404982629Z INFO: Collecting host service logs for NetworkManager
2026-07-24T11:32:30.405413581Z INFO: Collecting host service logs for openvswitch
2026-07-24T11:32:30.405902873Z INFO: Collecting host service logs for ovs-configuration
2026-07-24T11:32:30.406350711Z INFO: Collecting host service logs for ovsdb-server
2026-07-24T11:32:30.406700374Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-24T11:32:30.407118195Z INFO: Waiting for worker host service log collection to complete ...
2026-07-24T11:32:30.410498916Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-24T11:32:30.413116957Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-24T11:32:30.428310668Z INFO: Gathering HAProxy config files
2026-07-24T11:32:30.447254559Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-24T11:32:30.600151591Z INFO: Waiting for node performance related collection to complete ...
2026-07-24T11:32:30.939780479Z INFO: "metallb-operator" not detected. Skipping.
2026-07-24T11:32:31.096927890Z Wrote inspect data to must-gather.
2026-07-24T11:32:31.218828159Z No resources found in openshift-etcd namespace.
2026-07-24T11:32:31.601463759Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-24T11:32:31.631493433Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-24T11:32:31.704194773Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-24T11:32:31.807404127Z error: the server doesn't have a resource type "performanceprofile"
2026-07-24T11:32:31.929886669Z Wrote inspect data to must-gather.
2026-07-24T11:32:32.053285310Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-fk67d
2026-07-24T11:32:32.053344381Z insights-runtime-extractor-2tn4l
2026-07-24T11:32:32.053364933Z insights-runtime-extractor-jtwpm
2026-07-24T11:32:32.053399051Z insights-runtime-extractor-x5jn2
2026-07-24T11:32:32.077264244Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-24T11:32:32.202104008Z No resources found
2026-07-24T11:32:32.204368094Z ERROR: No running kube-apiserver pods found
2026-07-24T11:32:32.241966355Z INFO: Worker host service log collection to complete.
2026-07-24T11:32:32.268774406Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-24T11:32:32.269666368Z Wrote inspect data to must-gather.
2026-07-24T11:32:32.350401081Z Wrote inspect data to must-gather.
2026-07-24T11:32:32.503198727Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-24T11:32:32.503198727Z See 'oc rsync -h' for help and examples
2026-07-24T11:32:32.508318094Z No resources found
2026-07-24T11:32:32.595330857Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-24T11:32:32.595407906Z INFO: on-disk MachineConfig config collection complete.
2026-07-24T11:32:32.632465233Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-24T11:32:32.655532202Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-24T11:32:32.704263362Z Wrote inspect data to must-gather.
2026-07-24T11:32:32.715604636Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T11:32:32.823430135Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T11:32:32.871055366Z Wrote inspect data to must-gather.
2026-07-24T11:32:32.919511974Z Wrote inspect data to must-gather.
2026-07-24T11:32:32.944423402Z tar: Removing leading `/' from member names
2026-07-24T11:32:32.950985337Z INFO: HAProxy config collection complete.
2026-07-24T11:32:32.972047591Z error: the server doesn't have a resource type "machineconfigs"
2026-07-24T11:32:33.026302836Z INFO: 'previous-logs' folder not found on ip-10-0-131-78.ec2.internal, skipping...
2026-07-24T11:32:33.171781063Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-24T11:32:33.190862172Z Gathering data for ns/openshift-monitoring...
2026-07-24T11:32:33.217671626Z INFO: Getting rules from prometheus-k8s-0
2026-07-24T11:32:33.252841325Z Wrote inspect data to must-gather.
2026-07-24T11:32:33.298922189Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-24T11:32:33.352902423Z INFO: 'previous-logs' folder not found on ip-10-0-133-61.ec2.internal, skipping...
2026-07-24T11:32:33.420591889Z Wrote inspect data to must-gather.
2026-07-24T11:32:33.524948336Z Wrote inspect data to must-gather.
2026-07-24T11:32:33.601338648Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-24T11:32:33.603316079Z INFO: 'previous-logs' folder not found on ip-10-0-137-219.ec2.internal, skipping...
2026-07-24T11:32:33.603342424Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-24T11:32:33.603371847Z INFO: Machine Config Daemon termination log collection complete.
2026-07-24T11:32:33.663902387Z INFO: Getting status/config from prometheus-k8s-0
2026-07-24T11:32:33.784330762Z Wrote inspect data to must-gather.
2026-07-24T11:32:33.809649041Z Wrote inspect data to must-gather.
2026-07-24T11:32:34.003365467Z Wrote inspect data to must-gather.
2026-07-24T11:32:34.061049363Z INFO: INTERCONNECT MODE
2026-07-24T11:32:34.061085642Z INFO: Gathering ovn-kubernetes DBs
2026-07-24T11:32:34.076780947Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-24T11:32:34.084579747Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-24T11:32:34.229920091Z Wrote inspect data to must-gather.
2026-07-24T11:32:34.232535510Z INFO: Gathering OVN_Northbound from ovnkube-node-4vfr2...
2026-07-24T11:32:34.232781184Z INFO: Gathering OVN_Northbound from ovnkube-node-7j2qd...
2026-07-24T11:32:34.233064925Z INFO: Gathering OVN_Northbound from ovnkube-node-n7wb9...
2026-07-24T11:32:34.233303587Z INFO: Gathering OVN_Southbound from ovnkube-node-4vfr2...
2026-07-24T11:32:34.233531769Z INFO: Gathering OVN_Southbound from ovnkube-node-7j2qd...
2026-07-24T11:32:34.233751701Z INFO: Gathering OVN_Southbound from ovnkube-node-n7wb9...
2026-07-24T11:32:34.296765928Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:34.351164542Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-24T11:32:34.541482957Z tar: Removing leading `/' from member names
2026-07-24T11:32:34.546619530Z tar: Removing leading `/' from member names
2026-07-24T11:32:34.563323490Z tar: Removing leading `/' from member names
2026-07-24T11:32:34.585436803Z tar: Removing leading `/' from member names
2026-07-24T11:32:34.595723857Z Wrote inspect data to must-gather.
2026-07-24T11:32:34.614059035Z tar: Removing leading `/' from member names
2026-07-24T11:32:34.627476525Z tar: Removing leading `/' from member names
2026-07-24T11:32:34.658425645Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:34.660778568Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-24T11:32:34.682719043Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-24T11:32:34.821553642Z Wrote inspect data to must-gather.
2026-07-24T11:32:34.871233344Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:34.959542771Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-24T11:32:35.056554925Z Wrote inspect data to must-gather.
2026-07-24T11:32:35.075574674Z INFO: Waiting for network log collection to complete ...
2026-07-24T11:32:35.075626634Z INFO: Waiting for ovnk database copies to complete ...
2026-07-24T11:32:35.076141724Z INFO: Copying ovnk databases complete.
2026-07-24T11:32:35.079338854Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-24T11:32:35.081987759Z ovnk_database_store/
2026-07-24T11:32:35.081987759Z ovnk_database_store/ovnkube-node-7j2qd_nbdb
2026-07-24T11:32:35.091542819Z ovnk_database_store/ovnkube-node-n7wb9_sbdb
2026-07-24T11:32:35.108186707Z ovnk_database_store/ovnkube-node-4vfr2_sbdb
2026-07-24T11:32:35.117109835Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:35.119665278Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-24T11:32:35.123668310Z ovnk_database_store/ovnkube-node-n7wb9_nbdb
2026-07-24T11:32:35.128741348Z ovnk_database_store/ovnkube-node-4vfr2_nbdb
2026-07-24T11:32:35.134712249Z ovnk_database_store/ovnkube-node-7j2qd_sbdb
2026-07-24T11:32:35.144990118Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-24T11:32:35.155365418Z INFO: Network log collection complete.
2026-07-24T11:32:35.248187743Z Wrote inspect data to must-gather.
2026-07-24T11:32:35.308550313Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:35.390212352Z INFO: Getting status from alertmanager-main-0
2026-07-24T11:32:35.523519693Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-24T11:32:35.524301249Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:35.525375992Z INFO: Fallback to identify the container image from release info
2026-07-24T11:32:35.527703556Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-24T11:32:35.735955380Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:35.770333140Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-24T11:32:35.925752540Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:35.928023880Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-24T11:32:35.928542519Z Gathering data for ns/openshift-network-console...
2026-07-24T11:32:36.089393482Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:36.115327484Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-24T11:32:36.247570320Z daemonset.apps/perf-node-gather-daemonset created
2026-07-24T11:32:36.309093404Z Wrote inspect data to /must-gather/istio.
2026-07-24T11:32:36.387509740Z Gathering data for ns/openshift-console-operator...
2026-07-24T11:32:36.391430649Z Waiting for performance profile collector pods to become ready: 1
2026-07-24T11:32:36.778639526Z Gathering data for ns/openshift-console...
2026-07-24T11:32:36.982702584Z Done executing Istio gather script
2026-07-24T11:32:37.251367234Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-24T11:32:37.505106485Z Waiting for performance profile collector pods to become ready: 2
2026-07-24T11:32:37.727158476Z Gathering data for ns/openshift-dns-operator...
2026-07-24T11:32:37.828630437Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T11:32:37.990773595Z Gathering data for ns/openshift-dns...
2026-07-24T11:32:38.614434684Z Gathering data for ns/openshift-image-registry...
2026-07-24T11:32:38.631384684Z Waiting for performance profile collector pods to become ready: 3
2026-07-24T11:32:39.056916150Z Gathering data for ns/openshift-ingress-operator...
2026-07-24T11:32:39.352649925Z Gathering data for ns/openshift-ingress...
2026-07-24T11:32:39.652404654Z Gathering data for ns/openshift-ingress-canary...
2026-07-24T11:32:39.740399969Z Waiting for performance profile collector pods to become ready: 4
2026-07-24T11:32:40.035601995Z Gathering data for ns/openshift-insights...
2026-07-24T11:32:40.845159857Z Waiting for performance profile collector pods to become ready: 5
2026-07-24T11:32:41.821594152Z Gathering data for ns/openshift-jobset-operator...
2026-07-24T11:32:41.964600315Z Waiting for performance profile collector pods to become ready: 6
2026-07-24T11:32:42.589645861Z Gathering data for ns/openshift-config...
2026-07-24T11:32:42.833645992Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T11:32:42.846827509Z Gathering data for ns/openshift-config-managed...
2026-07-24T11:32:43.072179566Z Waiting for performance profile collector pods to become ready: 7
2026-07-24T11:32:43.308546905Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-24T11:32:43.552307214Z Gathering data for ns/openshift-kube-apiserver...
2026-07-24T11:32:43.793090150Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-24T11:32:44.058503003Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-24T11:32:44.186681912Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-24T11:32:44.361209176Z Gathering data for ns/openshift-kube-scheduler...
2026-07-24T11:32:44.435212948Z Collecting performance related data for node ip-10-0-133-61.ec2.internal
2026-07-24T11:32:44.444582496Z Collecting performance related data for node ip-10-0-137-219.ec2.internal
2026-07-24T11:32:44.456968927Z Collecting performance related data for node ip-10-0-131-78.ec2.internal
2026-07-24T11:32:44.695276531Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-24T11:32:45.068495108Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-24T11:32:45.490526691Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-24T11:32:45.824646423Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-24T11:32:46.392718013Z Gathering data for ns/openshift-multus...
2026-07-24T11:32:47.839293510Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T11:32:47.905691474Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-24T11:32:49.151830099Z Gathering data for ns/openshift-host-network...
2026-07-24T11:32:49.508080318Z Gathering data for ns/openshift-network-diagnostics...
2026-07-24T11:32:50.020880299Z Gathering data for ns/openshift-network-node-identity...
2026-07-24T11:32:50.401980674Z Gathering data for ns/openshift-network-operator...
2026-07-24T11:32:50.819209837Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-24T11:32:51.074248595Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-24T11:32:51.575264947Z Gathering data for ns/openshift-apiserver-operator...
2026-07-24T11:32:51.877943427Z Gathering data for ns/openshift-apiserver...
2026-07-24T11:32:52.314062987Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-24T11:32:52.494363573Z Collecting kubelet logs for node ip-10-0-133-61.ec2.internal
2026-07-24T11:32:52.496272533Z Collecting kubelet logs for node ip-10-0-137-219.ec2.internal
2026-07-24T11:32:52.498015605Z Collecting kubelet logs for node ip-10-0-131-78.ec2.internal
2026-07-24T11:32:52.684981803Z Gathering data for ns/openshift-controller-manager...
2026-07-24T11:32:52.844062435Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-24T11:32:52.852313377Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-t99jp namespace
2026-07-24T11:32:52.858626096Z INFO: Node performance data collection complete.
2026-07-24T11:32:52.994458052Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-24T11:32:53.634385306Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-24T11:32:53.902111934Z Gathering data for ns/openshift-service-ca-operator...
2026-07-24T11:32:54.199687164Z Gathering data for ns/openshift-service-ca...
2026-07-24T11:32:54.529314784Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-24T11:32:55.362686359Z Wrote inspect data to must-gather.
2026-07-24T11:32:55.362733505Z error: inspection completed with the errors occurred while gathering data:
2026-07-24T11:32:55.362733505Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-24T11:32:55.485565174Z error: the server doesn't have a resource type "clusters"
2026-07-24T11:32:55.617445567Z Caches written to disk