2026-05-29T20:39:07.949947562Z [disk usage checker] Started
2026-05-29T20:39:07.953270384Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-05-29T20:39:08.147930668Z Gathering data for ns/openshift-cluster-version...
2026-05-29T20:39:08.297882395Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T20:39:08.384526424Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T20:39:08.460386332Z Gathering data for ns/default...
2026-05-29T20:39:08.731866785Z Gathering data for ns/openshift...
2026-05-29T20:39:09.137074848Z Gathering data for ns/kube-system...
2026-05-29T20:39:09.861228784Z Gathering data for ns/openshift-etcd...
2026-05-29T20:39:10.168984707Z Wrote inspect data to must-gather.
2026-05-29T20:39:10.169015301Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T20:39:10.169015301Z     namespaces "assisted-installer" not found
2026-05-29T20:39:10.175000929Z Waiting on subprocesses to finish execution.
2026-05-29T20:39:10.184721575Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-29T20:39:10.185564614Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-29T20:39:10.190288306Z INFO: Collecting host service logs for crio
2026-05-29T20:39:10.190652847Z INFO: Collecting host service logs for kubelet
2026-05-29T20:39:10.190960621Z INFO: Collecting host service logs for rpm-ostreed
2026-05-29T20:39:10.191252414Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-29T20:39:10.191601204Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-29T20:39:10.191897404Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-29T20:39:10.192246785Z INFO: Collecting host service logs for NetworkManager
2026-05-29T20:39:10.192702163Z INFO: Collecting host service logs for openvswitch
2026-05-29T20:39:10.193038080Z INFO: Collecting host service logs for ovs-configuration
2026-05-29T20:39:10.193387972Z INFO: Collecting host service logs for ovsdb-server
2026-05-29T20:39:10.193745426Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-29T20:39:10.194148832Z INFO: Waiting for worker host service log collection to complete ...
2026-05-29T20:39:10.194423465Z INFO: Gathering HAProxy config files
2026-05-29T20:39:10.195175139Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-29T20:39:10.218377772Z INFO: Waiting for node performance related collection to complete ...
2026-05-29T20:39:10.827508717Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-29T20:39:10.875621412Z INFO: "metallb-operator" not detected. Skipping.
2026-05-29T20:39:10.897838876Z INFO: Collecting Insights Archives from insights-operator-788bf58df5-c26bf
2026-05-29T20:39:10.897838876Z insights-runtime-extractor-ch8pw
2026-05-29T20:39:10.897838876Z insights-runtime-extractor-gshzz
2026-05-29T20:39:10.897838876Z insights-runtime-extractor-mkwv5
2026-05-29T20:39:10.923335994Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-29T20:39:11.010082828Z error: the server doesn't have a resource type "performanceprofile"
2026-05-29T20:39:11.168445301Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-29T20:39:11.168538727Z INFO: on-disk MachineConfig config collection complete.
2026-05-29T20:39:11.266064215Z No resources found in openshift-etcd namespace.
2026-05-29T20:39:11.335470607Z No resources found
2026-05-29T20:39:11.368507785Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-29T20:39:11.374330579Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-29T20:39:11.454671336Z Wrote inspect data to must-gather.
2026-05-29T20:39:11.608704543Z Wrote inspect data to must-gather.
2026-05-29T20:39:11.714995127Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-29T20:39:11.714995127Z See 'oc rsync -h' for help and examples
2026-05-29T20:39:11.802621810Z Wrote inspect data to must-gather.
2026-05-29T20:39:11.869331275Z Wrote inspect data to must-gather.
2026-05-29T20:39:11.960517972Z INFO: Worker host service log collection to complete.
2026-05-29T20:39:11.964894560Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-29T20:39:12.039721657Z No resources found
2026-05-29T20:39:12.106149748Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-29T20:39:12.126640939Z Wrote inspect data to must-gather.
2026-05-29T20:39:12.152824965Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-29T20:39:12.161366503Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T20:39:12.229070006Z error: the server doesn't have a resource type "machineconfigs"
2026-05-29T20:39:12.250618923Z Wrote inspect data to must-gather.
2026-05-29T20:39:12.259990995Z ERROR: No running kube-apiserver pods found
2026-05-29T20:39:12.316006738Z tar: Removing leading `/' from member names
2026-05-29T20:39:12.321301309Z INFO: HAProxy config collection complete.
2026-05-29T20:39:12.351940563Z Wrote inspect data to must-gather.
2026-05-29T20:39:12.384196715Z INFO: 'previous-logs' folder not found on ip-10-0-129-45.ec2.internal, skipping...
2026-05-29T20:39:12.436100689Z Gathering data for ns/openshift-monitoring...
2026-05-29T20:39:12.560332627Z Wrote inspect data to must-gather.
2026-05-29T20:39:12.615830329Z Wrote inspect data to must-gather.
2026-05-29T20:39:12.616918192Z INFO: Getting rules from prometheus-k8s-0
2026-05-29T20:39:12.684458579Z INFO: 'previous-logs' folder not found on ip-10-0-141-51.ec2.internal, skipping...
2026-05-29T20:39:12.812323407Z Wrote inspect data to must-gather.
2026-05-29T20:39:12.904554981Z Wrote inspect data to must-gather.
2026-05-29T20:39:12.930995003Z INFO: 'previous-logs' folder not found on ip-10-0-142-214.ec2.internal, skipping...
2026-05-29T20:39:12.931030438Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-29T20:39:12.931037911Z INFO: Machine Config Daemon termination log collection complete.
2026-05-29T20:39:12.959590141Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-05-29T20:39:12.961684942Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-29T20:39:12.999720556Z INFO: Getting status/config from prometheus-k8s-0
2026-05-29T20:39:13.145525844Z Wrote inspect data to must-gather.
2026-05-29T20:39:13.168047237Z Wrote inspect data to must-gather.
2026-05-29T20:39:13.357452674Z Wrote inspect data to must-gather.
2026-05-29T20:39:13.365582152Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-29T20:39:13.405828851Z INFO: INTERCONNECT MODE
2026-05-29T20:39:13.405912211Z INFO: Gathering ovn-kubernetes DBs
2026-05-29T20:39:13.542185954Z Wrote inspect data to must-gather.
2026-05-29T20:39:13.550685534Z INFO: Gathering OVN_Northbound from ovnkube-node-jxvss...
2026-05-29T20:39:13.550933445Z INFO: Gathering OVN_Northbound from ovnkube-node-qs86t...
2026-05-29T20:39:13.551174929Z INFO: Gathering OVN_Northbound from ovnkube-node-rhjcn...
2026-05-29T20:39:13.551501707Z INFO: Gathering OVN_Southbound from ovnkube-node-jxvss...
2026-05-29T20:39:13.551746391Z INFO: Gathering OVN_Southbound from ovnkube-node-qs86t...
2026-05-29T20:39:13.552035752Z INFO: Gathering OVN_Southbound from ovnkube-node-rhjcn...
2026-05-29T20:39:13.593514220Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-29T20:39:13.844140150Z tar: Removing leading `/' from member names
2026-05-29T20:39:13.873525306Z tar: Removing leading `/' from member names
2026-05-29T20:39:13.873851665Z tar: Removing leading `/' from member names
2026-05-29T20:39:13.885583477Z tar: Removing leading `/' from member names
2026-05-29T20:39:13.886789555Z Wrote inspect data to must-gather.
2026-05-29T20:39:13.895615636Z tar: Removing leading `/' from member names
2026-05-29T20:39:13.903218081Z tar: Removing leading `/' from member names
2026-05-29T20:39:13.924564374Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T20:39:13.957944066Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-29T20:39:14.096260413Z Wrote inspect data to must-gather.
2026-05-29T20:39:14.222344348Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-29T20:39:14.324390452Z Wrote inspect data to must-gather.
2026-05-29T20:39:14.340865345Z INFO: Waiting for network log collection to complete ...
2026-05-29T20:39:14.340901288Z INFO: Waiting for ovnk database copies to complete ...
2026-05-29T20:39:14.341345860Z INFO: Copying ovnk databases complete.
2026-05-29T20:39:14.343145192Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-05-29T20:39:14.345206621Z ovnk_database_store/
2026-05-29T20:39:14.345206621Z ovnk_database_store/ovnkube-node-rhjcn_nbdb
2026-05-29T20:39:14.351876772Z ovnk_database_store/ovnkube-node-jxvss_sbdb
2026-05-29T20:39:14.366963642Z ovnk_database_store/ovnkube-node-jxvss_nbdb
2026-05-29T20:39:14.374743267Z ovnk_database_store/ovnkube-node-qs86t_nbdb
2026-05-29T20:39:14.381307831Z ovnk_database_store/ovnkube-node-qs86t_sbdb
2026-05-29T20:39:14.399330866Z ovnk_database_store/ovnkube-node-rhjcn_sbdb
2026-05-29T20:39:14.415254043Z INFO: Network log collection complete.
2026-05-29T20:39:14.517604141Z W0529 20:39:14.517550    1402 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-29T20:39:14.524984501Z Wrote inspect data to must-gather.
2026-05-29T20:39:14.561479847Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T20:39:14.575761521Z INFO: Getting status from alertmanager-main-0
2026-05-29T20:39:14.735066257Z Gathering data for ns/openshift-network-console...
2026-05-29T20:39:14.779836644Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-29T20:39:14.782525442Z INFO: Fallback to identify the container image from release info
2026-05-29T20:39:15.108874241Z Gathering data for ns/openshift-console-operator...
2026-05-29T20:39:15.463657650Z Gathering data for ns/openshift-console...
2026-05-29T20:39:15.479384348Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4d8381cb85e479760bd6b601e111b39e6c56a632bfa298867f4197ec4421e9f5
2026-05-29T20:39:15.602918789Z daemonset.apps/perf-node-gather-daemonset created
2026-05-29T20:39:15.709133418Z Waiting for performance profile collector pods to become ready: 1
2026-05-29T20:39:15.902281888Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-29T20:39:16.251600858Z Gathering data for ns/openshift-dns-operator...
2026-05-29T20:39:16.506568958Z Gathering data for ns/openshift-dns...
2026-05-29T20:39:16.824313965Z Waiting for performance profile collector pods to become ready: 2
2026-05-29T20:39:17.071608508Z Gathering data for ns/openshift-image-registry...
2026-05-29T20:39:17.505644403Z Gathering data for ns/openshift-ingress-operator...
2026-05-29T20:39:17.807975461Z Gathering data for ns/openshift-ingress...
2026-05-29T20:39:17.940685023Z Waiting for performance profile collector pods to become ready: 3
2026-05-29T20:39:17.964442739Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-05-29T20:39:18.120329804Z Gathering data for ns/openshift-ingress-canary...
2026-05-29T20:39:18.471806403Z Gathering data for ns/openshift-insights...
2026-05-29T20:39:19.059233752Z Waiting for performance profile collector pods to become ready: 4
2026-05-29T20:39:20.165889060Z Waiting for performance profile collector pods to become ready: 5
2026-05-29T20:39:20.205333526Z Gathering data for ns/openshift-config...
2026-05-29T20:39:20.463197529Z Gathering data for ns/openshift-config-managed...
2026-05-29T20:39:20.870636417Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-29T20:39:21.132317881Z Gathering data for ns/openshift-kube-apiserver...
2026-05-29T20:39:21.275281310Z Waiting for performance profile collector pods to become ready: 6
2026-05-29T20:39:21.409845240Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-29T20:39:21.692903612Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-29T20:39:21.962800088Z Gathering data for ns/openshift-kube-scheduler...
2026-05-29T20:39:22.264129692Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-29T20:39:22.392262841Z Waiting for performance profile collector pods to become ready: 7
2026-05-29T20:39:22.557345404Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-29T20:39:22.871841693Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-29T20:39:22.968900672Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-05-29T20:39:23.173969051Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-29T20:39:23.508729169Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-29T20:39:23.678912223Z Gathering data for ns/openshift-multus...
2026-05-29T20:39:23.753826850Z Collecting performance related data for node ip-10-0-142-214.ec2.internal
2026-05-29T20:39:23.760744490Z Collecting performance related data for node ip-10-0-141-51.ec2.internal
2026-05-29T20:39:23.767460840Z Collecting performance related data for node ip-10-0-129-45.ec2.internal
2026-05-29T20:39:25.256323148Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-29T20:39:26.752933215Z Gathering data for ns/openshift-host-network...
2026-05-29T20:39:27.053701536Z Gathering data for ns/openshift-network-diagnostics...
2026-05-29T20:39:27.560642213Z Gathering data for ns/openshift-network-node-identity...
2026-05-29T20:39:27.941169318Z Gathering data for ns/openshift-network-operator...
2026-05-29T20:39:27.973473087Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-05-29T20:39:28.319549048Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-29T20:39:28.604684956Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-29T20:39:29.099285946Z Gathering data for ns/openshift-apiserver-operator...
2026-05-29T20:39:29.391103401Z Gathering data for ns/openshift-apiserver...
2026-05-29T20:39:29.817217895Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-29T20:39:30.084776893Z Gathering data for ns/openshift-controller-manager...
2026-05-29T20:39:30.362774660Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-29T20:39:31.037614265Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-29T20:39:31.350099221Z Gathering data for ns/openshift-service-ca-operator...
2026-05-29T20:39:31.670115210Z Gathering data for ns/openshift-service-ca...
2026-05-29T20:39:31.687905889Z Collecting kubelet logs for node ip-10-0-142-214.ec2.internal
2026-05-29T20:39:31.689868501Z Collecting kubelet logs for node ip-10-0-141-51.ec2.internal
2026-05-29T20:39:31.692053585Z Collecting kubelet logs for node ip-10-0-129-45.ec2.internal
2026-05-29T20:39:32.044966847Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-29T20:39:32.050206182Z INFO: Node performance data collection complete.
2026-05-29T20:39:32.086863747Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-29T20:39:32.977881570Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-05-29T20:39:32.987931245Z Wrote inspect data to must-gather.
2026-05-29T20:39:32.987966213Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T20:39:32.987966213Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-29T20:39:33.113103583Z error: the server doesn't have a resource type "clusters"
2026-05-29T20:39:33.226186590Z Caches written to disk