2026-07-31T17:02:29.549572551Z [disk usage checker] Started
2026-07-31T17:02:29.553533349Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-07-31T17:02:29.746695760Z Gathering data for ns/openshift-cluster-version...
2026-07-31T17:02:29.875200863Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T17:02:29.960023558Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T17:02:30.039682355Z Gathering data for ns/default...
2026-07-31T17:02:30.321971341Z Gathering data for ns/openshift...
2026-07-31T17:02:30.771384021Z Gathering data for ns/kube-system...
2026-07-31T17:02:31.426226639Z Gathering data for ns/openshift-etcd...
2026-07-31T17:02:31.715794000Z Wrote inspect data to must-gather.
2026-07-31T17:02:31.715830912Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T17:02:31.715830912Z     namespaces "assisted-installer" not found
2026-07-31T17:02:32.119317066Z Waiting on subprocesses to finish execution.
2026-07-31T17:02:32.129801413Z INFO: Gathering HAProxy config files
2026-07-31T17:02:32.130619160Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-31T17:02:32.131835381Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-31T17:02:32.140572274Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-31T17:02:32.181775466Z INFO: Collecting host service logs for crio
2026-07-31T17:02:32.182143098Z INFO: Collecting host service logs for kubelet
2026-07-31T17:02:32.182505691Z INFO: Collecting host service logs for rpm-ostreed
2026-07-31T17:02:32.182822805Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-31T17:02:32.183139989Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-31T17:02:32.183481825Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-31T17:02:32.183788567Z INFO: Collecting host service logs for NetworkManager
2026-07-31T17:02:32.184084615Z INFO: Collecting host service logs for openvswitch
2026-07-31T17:02:32.184381016Z INFO: Collecting host service logs for ovs-configuration
2026-07-31T17:02:32.184705492Z INFO: Collecting host service logs for ovsdb-server
2026-07-31T17:02:32.185016147Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-31T17:02:32.185374721Z INFO: Waiting for worker host service log collection to complete ...
2026-07-31T17:02:32.190718016Z Executing Istio gather script
2026-07-31T17:02:32.193088855Z INFO: Waiting for node performance related collection to complete ...
2026-07-31T17:02:32.550535136Z INFO: Collecting Insights Archives from insights-operator-77696685b-cjrm5
2026-07-31T17:02:32.550591704Z insights-runtime-extractor-8cxg8
2026-07-31T17:02:32.550612435Z insights-runtime-extractor-ffstb
2026-07-31T17:02:32.550632289Z insights-runtime-extractor-fxz5l
2026-07-31T17:02:33.110046071Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-31T17:02:33.110083825Z INFO: on-disk MachineConfig config collection complete.
2026-07-31T17:02:33.190133668Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-31T17:02:33.428989809Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-31T17:02:33.437109694Z INFO: "metallb-operator" not detected. Skipping.
2026-07-31T17:02:33.470528800Z No resources found
2026-07-31T17:02:33.509038088Z No resources found in openshift-etcd namespace.
2026-07-31T17:02:33.510865289Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-31T17:02:33.655744632Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T17:02:33.763715619Z error: the server doesn't have a resource type "performanceprofile"
2026-07-31T17:02:33.912073170Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-31T17:02:34.016750231Z No resources found
2026-07-31T17:02:34.122908416Z Gathering data for ns/openshift-monitoring...
2026-07-31T17:02:34.134033162Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-31T17:02:34.134033162Z See 'oc rsync -h' for help and examples
2026-07-31T17:02:34.192921425Z Wrote inspect data to must-gather.
2026-07-31T17:02:34.346392552Z INFO: Worker host service log collection to complete.
2026-07-31T17:02:34.451986898Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-31T17:02:34.532592150Z ERROR: No running kube-apiserver pods found
2026-07-31T17:02:34.572356315Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-31T17:02:34.590726897Z INFO: 'previous-logs' folder not found on ip-10-0-129-212.ec2.internal, skipping...
2026-07-31T17:02:34.599217162Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-07-31T17:02:34.631932151Z Wrote inspect data to must-gather.
2026-07-31T17:02:34.660519164Z Wrote inspect data to must-gather.
2026-07-31T17:02:34.680417808Z Wrote inspect data to must-gather.
2026-07-31T17:02:34.697177755Z Wrote inspect data to must-gather.
2026-07-31T17:02:34.753585144Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-31T17:02:34.832238255Z Wrote inspect data to must-gather.
2026-07-31T17:02:34.855486350Z Wrote inspect data to must-gather.
2026-07-31T17:02:34.909724086Z tar: Removing leading `/' from member names
2026-07-31T17:02:34.919686613Z INFO: HAProxy config collection complete.
2026-07-31T17:02:34.939816540Z error: the server doesn't have a resource type "machineconfigs"
2026-07-31T17:02:34.968004716Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-31T17:02:35.005558723Z INFO: 'previous-logs' folder not found on ip-10-0-140-127.ec2.internal, skipping...
2026-07-31T17:02:35.130888722Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-31T17:02:35.153249109Z Wrote inspect data to must-gather.
2026-07-31T17:02:35.271258694Z INFO: 'previous-logs' folder not found on ip-10-0-140-34.ec2.internal, skipping...
2026-07-31T17:02:35.271289723Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-31T17:02:35.271320884Z INFO: Machine Config Daemon termination log collection complete.
2026-07-31T17:02:35.280522430Z INFO: Getting rules from prometheus-k8s-0
2026-07-31T17:02:35.357798507Z Wrote inspect data to must-gather.
2026-07-31T17:02:35.426066746Z Wrote inspect data to must-gather.
2026-07-31T17:02:35.515193143Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-31T17:02:35.665223948Z Wrote inspect data to must-gather.
2026-07-31T17:02:35.717015590Z Wrote inspect data to must-gather.
2026-07-31T17:02:35.735841984Z INFO: Getting status/config from prometheus-k8s-0
2026-07-31T17:02:35.953739249Z Wrote inspect data to must-gather.
2026-07-31T17:02:35.978229750Z INFO: INTERCONNECT MODE
2026-07-31T17:02:35.978274122Z INFO: Gathering ovn-kubernetes DBs
2026-07-31T17:02:36.148690782Z INFO: Gathering OVN_Northbound from ovnkube-node-6sdns...
2026-07-31T17:02:36.148918544Z INFO: Gathering OVN_Northbound from ovnkube-node-dwr22...
2026-07-31T17:02:36.149124543Z INFO: Gathering OVN_Northbound from ovnkube-node-fn5jp...
2026-07-31T17:02:36.149426681Z INFO: Gathering OVN_Southbound from ovnkube-node-6sdns...
2026-07-31T17:02:36.149742290Z INFO: Gathering OVN_Southbound from ovnkube-node-dwr22...
2026-07-31T17:02:36.150053016Z INFO: Gathering OVN_Southbound from ovnkube-node-fn5jp...
2026-07-31T17:02:36.167973304Z Wrote inspect data to must-gather.
2026-07-31T17:02:36.369052466Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-31T17:02:36.446479064Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-31T17:02:36.447276143Z tar: Removing leading `/' from member names
2026-07-31T17:02:36.487874472Z tar: Removing leading `/' from member names
2026-07-31T17:02:36.498646000Z tar: Removing leading `/' from member names
2026-07-31T17:02:36.518789639Z tar: Removing leading `/' from member names
2026-07-31T17:02:36.546435454Z tar: Removing leading `/' from member names
2026-07-31T17:02:36.553956745Z Wrote inspect data to must-gather.
2026-07-31T17:02:36.556618734Z tar: Removing leading `/' from member names
2026-07-31T17:02:36.589673637Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T17:02:36.684040361Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:36.713468624Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-31T17:02:36.836112596Z Wrote inspect data to must-gather.
2026-07-31T17:02:36.934177548Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:36.937481814Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-31T17:02:37.000496034Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-31T17:02:37.004729643Z INFO: Waiting for network log collection to complete ...
2026-07-31T17:02:37.004781223Z INFO: Waiting for ovnk database copies to complete ...
2026-07-31T17:02:37.005295009Z INFO: Copying ovnk databases complete.
2026-07-31T17:02:37.007174917Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-07-31T17:02:37.009913022Z ovnk_database_store/
2026-07-31T17:02:37.009913022Z ovnk_database_store/ovnkube-node-6sdns_nbdb
2026-07-31T17:02:37.019063273Z ovnk_database_store/ovnkube-node-fn5jp_nbdb
2026-07-31T17:02:37.026890281Z ovnk_database_store/ovnkube-node-6sdns_sbdb
2026-07-31T17:02:37.048240348Z ovnk_database_store/ovnkube-node-fn5jp_sbdb
2026-07-31T17:02:37.066906167Z ovnk_database_store/ovnkube-node-dwr22_sbdb
2026-07-31T17:02:37.081859851Z Wrote inspect data to must-gather.
2026-07-31T17:02:37.086369896Z ovnk_database_store/ovnkube-node-dwr22_nbdb
2026-07-31T17:02:37.097993255Z INFO: Network log collection complete.
2026-07-31T17:02:37.160931645Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:37.266190389Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-31T17:02:37.288009499Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T17:02:37.395199162Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:37.397760354Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-31T17:02:37.405620030Z Wrote inspect data to must-gather.
2026-07-31T17:02:37.482808453Z Gathering data for ns/openshift-network-console...
2026-07-31T17:02:37.614762555Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:37.693819590Z INFO: Getting status from alertmanager-main-0
2026-07-31T17:02:37.696670395Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-31T17:02:37.699362227Z INFO: Fallback to identify the container image from release info
2026-07-31T17:02:37.841955837Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:37.845168835Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-31T17:02:37.973461442Z Gathering data for ns/openshift-console-operator...
2026-07-31T17:02:38.062525017Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:38.245148201Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:38.248595431Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-31T17:02:38.317819217Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-07-31T17:02:38.363725310Z Gathering data for ns/openshift-console...
2026-07-31T17:02:38.423511725Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:38.448028017Z daemonset.apps/perf-node-gather-daemonset created
2026-07-31T17:02:38.587909770Z Waiting for performance profile collector pods to become ready: 1
2026-07-31T17:02:38.643320555Z Wrote inspect data to /must-gather/istio.
2026-07-31T17:02:38.960067311Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-31T17:02:39.338045070Z Done executing Istio gather script
2026-07-31T17:02:39.456462559Z Gathering data for ns/openshift-dns-operator...
2026-07-31T17:02:39.625126508Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-07-31T17:02:39.715783164Z Waiting for performance profile collector pods to become ready: 2
2026-07-31T17:02:39.732530957Z Gathering data for ns/openshift-dns...
2026-07-31T17:02:40.319364876Z Gathering data for ns/openshift-image-registry...
2026-07-31T17:02:40.773498529Z Gathering data for ns/openshift-ingress-operator...
2026-07-31T17:02:40.850036021Z Waiting for performance profile collector pods to become ready: 3
2026-07-31T17:02:41.089805114Z Gathering data for ns/openshift-ingress...
2026-07-31T17:02:41.404880258Z Gathering data for ns/openshift-ingress-canary...
2026-07-31T17:02:41.764528660Z Gathering data for ns/openshift-insights...
2026-07-31T17:02:41.951085973Z Waiting for performance profile collector pods to become ready: 4
2026-07-31T17:02:43.066301235Z Waiting for performance profile collector pods to become ready: 5
2026-07-31T17:02:43.575097802Z Gathering data for ns/openshift-jobset-operator...
2026-07-31T17:02:44.178848244Z Waiting for performance profile collector pods to become ready: 6
2026-07-31T17:02:44.352134320Z Gathering data for ns/openshift-config...
2026-07-31T17:02:44.630954338Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-07-31T17:02:44.639133809Z Gathering data for ns/openshift-config-managed...
2026-07-31T17:02:45.037285376Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-31T17:02:45.302551323Z Waiting for performance profile collector pods to become ready: 7
2026-07-31T17:02:45.350143879Z Gathering data for ns/openshift-kube-apiserver...
2026-07-31T17:02:45.613995503Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-31T17:02:45.940776271Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-31T17:02:46.259978123Z Gathering data for ns/openshift-kube-scheduler...
2026-07-31T17:02:46.410634596Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-31T17:02:46.535019554Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-31T17:02:46.672223429Z Collecting performance related data for node ip-10-0-129-212.ec2.internal
2026-07-31T17:02:46.679560358Z Collecting performance related data for node ip-10-0-140-34.ec2.internal
2026-07-31T17:02:46.689082654Z Collecting performance related data for node ip-10-0-140-127.ec2.internal
2026-07-31T17:02:46.871319998Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-31T17:02:47.264710780Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-31T17:02:47.797356129Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-31T17:02:48.391798233Z Gathering data for ns/openshift-multus...
2026-07-31T17:02:49.636059910Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-07-31T17:02:49.858080608Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-31T17:02:51.263821391Z Gathering data for ns/openshift-host-network...
2026-07-31T17:02:51.573032727Z Gathering data for ns/openshift-network-diagnostics...
2026-07-31T17:02:52.115881842Z Gathering data for ns/openshift-network-node-identity...
2026-07-31T17:02:52.512399000Z Gathering data for ns/openshift-network-operator...
2026-07-31T17:02:52.910946854Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-31T17:02:53.196518412Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-31T17:02:53.716882579Z Gathering data for ns/openshift-apiserver-operator...
2026-07-31T17:02:54.022689118Z Gathering data for ns/openshift-apiserver...
2026-07-31T17:02:54.418438770Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-31T17:02:54.640777279Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-07-31T17:02:54.692499538Z Gathering data for ns/openshift-controller-manager...
2026-07-31T17:02:54.956372997Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-31T17:02:55.007728315Z Collecting kubelet logs for node ip-10-0-129-212.ec2.internal
2026-07-31T17:02:55.009377014Z Collecting kubelet logs for node ip-10-0-140-34.ec2.internal
2026-07-31T17:02:55.011543802Z Collecting kubelet logs for node ip-10-0-140-127.ec2.internal
2026-07-31T17:02:55.373198949Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-j7xss namespace
2026-07-31T17:02:55.379488200Z INFO: Node performance data collection complete.
2026-07-31T17:02:55.670712786Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-31T17:02:55.956411485Z Gathering data for ns/openshift-service-ca-operator...
2026-07-31T17:02:56.286543201Z Gathering data for ns/openshift-service-ca...
2026-07-31T17:02:56.686811129Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-31T17:02:57.562194410Z Wrote inspect data to must-gather.
2026-07-31T17:02:57.562238803Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T17:02:57.562238803Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-31T17:02:57.686324361Z error: the server doesn't have a resource type "clusters"
2026-07-31T17:02:57.817099984Z Caches written to disk