2026-04-20T12:59:26.552178817Z [disk usage checker] Started
2026-04-20T12:59:26.555933957Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T12:59:26.755085105Z Gathering data for ns/openshift-cluster-version...
2026-04-20T12:59:26.880695643Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T12:59:26.962970556Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T12:59:27.039621228Z Gathering data for ns/default...
2026-04-20T12:59:27.317238035Z Gathering data for ns/openshift...
2026-04-20T12:59:27.695206375Z Gathering data for ns/kube-system...
2026-04-20T12:59:28.347561018Z Gathering data for ns/openshift-etcd...
2026-04-20T12:59:28.626012733Z Wrote inspect data to must-gather.
2026-04-20T12:59:28.626041014Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T12:59:28.626041014Z     namespaces "assisted-installer" not found
2026-04-20T12:59:28.735481390Z Waiting on subprocesses to finish execution.
2026-04-20T12:59:28.739928964Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-20T12:59:28.747186531Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-20T12:59:28.754987047Z INFO: Waiting for node performance related collection to complete ...
2026-04-20T12:59:28.759457472Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-20T12:59:28.769150660Z INFO: Collecting host service logs for crio
2026-04-20T12:59:28.769435876Z INFO: Collecting host service logs for kubelet
2026-04-20T12:59:28.769746709Z INFO: Collecting host service logs for rpm-ostreed
2026-04-20T12:59:28.770045168Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-20T12:59:28.770349425Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-20T12:59:28.770719712Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-20T12:59:28.771065308Z INFO: Collecting host service logs for NetworkManager
2026-04-20T12:59:28.771402485Z INFO: Collecting host service logs for openvswitch
2026-04-20T12:59:28.771772273Z INFO: Collecting host service logs for ovs-configuration
2026-04-20T12:59:28.772123027Z INFO: Collecting host service logs for ovsdb-server
2026-04-20T12:59:28.772458875Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-20T12:59:28.772878219Z INFO: Waiting for worker host service log collection to complete ...
2026-04-20T12:59:28.774864312Z INFO: Gathering HAProxy config files
2026-04-20T12:59:29.146479619Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-hlmfm
2026-04-20T12:59:29.146540201Z insights-runtime-extractor-54t5x
2026-04-20T12:59:29.146560467Z insights-runtime-extractor-6kp8w
2026-04-20T12:59:29.146579643Z insights-runtime-extractor-kxx9j
2026-04-20T12:59:29.556609515Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-20T12:59:29.556715374Z INFO: on-disk MachineConfig config collection complete.
2026-04-20T12:59:29.564317581Z No resources found
2026-04-20T12:59:29.821430330Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-20T12:59:29.944590303Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-20T12:59:29.964366432Z No resources found in openshift-etcd namespace.
2026-04-20T12:59:29.973466425Z error: the server doesn't have a resource type "performanceprofile"
2026-04-20T12:59:30.153501398Z Wrote inspect data to must-gather.
2026-04-20T12:59:30.210322513Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-20T12:59:30.210322513Z See 'oc rsync -h' for help and examples
2026-04-20T12:59:30.228766473Z INFO: Worker host service log collection to complete.
2026-04-20T12:59:30.283971342Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-20T12:59:30.321798993Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-20T12:59:30.500517943Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-20T12:59:30.504220091Z No resources found
2026-04-20T12:59:30.560964785Z INFO: "metallb-operator" not detected. Skipping.
2026-04-20T12:59:30.657051877Z Wrote inspect data to must-gather.
2026-04-20T12:59:30.749331856Z Wrote inspect data to must-gather.
2026-04-20T12:59:30.755043694Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-20T12:59:30.840238791Z Wrote inspect data to must-gather.
2026-04-20T12:59:30.862371408Z ERROR: No running kube-apiserver pods found
2026-04-20T12:59:30.869794625Z INFO: 'previous-logs' folder not found on ip-10-0-131-55.ec2.internal, skipping...
2026-04-20T12:59:30.878998180Z Wrote inspect data to must-gather.
2026-04-20T12:59:30.888282982Z Wrote inspect data to must-gather.
2026-04-20T12:59:30.907433652Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T12:59:30.924338172Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-20T12:59:31.031624988Z Gathering data for ns/openshift-monitoring...
2026-04-20T12:59:31.107248675Z Wrote inspect data to must-gather.
2026-04-20T12:59:31.115012518Z error: the server doesn't have a resource type "machineconfigs"
2026-04-20T12:59:31.119907695Z Wrote inspect data to must-gather.
2026-04-20T12:59:31.131608494Z tar: Removing leading `/' from member names
2026-04-20T12:59:31.137283024Z INFO: HAProxy config collection complete.
2026-04-20T12:59:31.250827478Z INFO: 'previous-logs' folder not found on ip-10-0-135-187.ec2.internal, skipping...
2026-04-20T12:59:31.347295648Z Wrote inspect data to must-gather.
2026-04-20T12:59:31.365205030Z Wrote inspect data to must-gather.
2026-04-20T12:59:31.370630855Z INFO: Getting rules from prometheus-k8s-0
2026-04-20T12:59:31.514600054Z INFO: 'previous-logs' folder not found on ip-10-0-137-91.ec2.internal, skipping...
2026-04-20T12:59:31.514660591Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-20T12:59:31.514683086Z INFO: Machine Config Daemon termination log collection complete.
2026-04-20T12:59:31.560833851Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T12:59:31.579683184Z Wrote inspect data to must-gather.
2026-04-20T12:59:31.598866296Z Wrote inspect data to must-gather.
2026-04-20T12:59:31.731538886Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-20T12:59:31.772570574Z INFO: Getting status/config from prometheus-k8s-0
2026-04-20T12:59:31.863679492Z INFO: INTERCONNECT MODE
2026-04-20T12:59:31.863728512Z INFO: Gathering ovn-kubernetes DBs
2026-04-20T12:59:31.933556458Z Wrote inspect data to must-gather.
2026-04-20T12:59:31.997525862Z INFO: Gathering OVN_Northbound from ovnkube-node-dwphh...
2026-04-20T12:59:31.997761320Z INFO: Gathering OVN_Northbound from ovnkube-node-pw7dn...
2026-04-20T12:59:31.997999815Z INFO: Gathering OVN_Northbound from ovnkube-node-zmhl5...
2026-04-20T12:59:31.998313543Z INFO: Gathering OVN_Southbound from ovnkube-node-dwphh...
2026-04-20T12:59:31.998553739Z INFO: Gathering OVN_Southbound from ovnkube-node-pw7dn...
2026-04-20T12:59:31.999001563Z INFO: Gathering OVN_Southbound from ovnkube-node-zmhl5...
2026-04-20T12:59:32.248605783Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-20T12:59:32.279432238Z Wrote inspect data to must-gather.
2026-04-20T12:59:32.297705406Z tar: Removing leading `/' from member names
2026-04-20T12:59:32.304808966Z tar: Removing leading `/' from member names
2026-04-20T12:59:32.304951867Z tar: Removing leading `/' from member names
2026-04-20T12:59:32.320844366Z tar: Removing leading `/' from member names
2026-04-20T12:59:32.322855992Z tar: Removing leading `/' from member names
2026-04-20T12:59:32.348963890Z tar: Removing leading `/' from member names
2026-04-20T12:59:32.429087863Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T12:59:32.495153716Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-20T12:59:32.515964489Z Wrote inspect data to must-gather.
2026-04-20T12:59:32.725439321Z Wrote inspect data to must-gather.
2026-04-20T12:59:32.750432540Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-20T12:59:32.766138873Z INFO: Waiting for network log collection to complete ...
2026-04-20T12:59:32.766190501Z INFO: Waiting for ovnk database copies to complete ...
2026-04-20T12:59:32.766709497Z INFO: Copying ovnk databases complete.
2026-04-20T12:59:32.768996995Z 3.1M	must-gather/network_logs/ovnk_database_store
2026-04-20T12:59:32.772628760Z ovnk_database_store/
2026-04-20T12:59:32.772628760Z ovnk_database_store/ovnkube-node-dwphh_nbdb
2026-04-20T12:59:32.781880056Z ovnk_database_store/ovnkube-node-zmhl5_sbdb
2026-04-20T12:59:32.798909146Z ovnk_database_store/ovnkube-node-pw7dn_sbdb
2026-04-20T12:59:32.815908663Z ovnk_database_store/ovnkube-node-dwphh_sbdb
2026-04-20T12:59:32.837288945Z ovnk_database_store/ovnkube-node-pw7dn_nbdb
2026-04-20T12:59:32.848199090Z ovnk_database_store/ovnkube-node-zmhl5_nbdb
2026-04-20T12:59:32.860925893Z INFO: Network log collection complete.
2026-04-20T12:59:32.921755657Z W0420 12:59:32.921717    1313 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-20T12:59:32.926553687Z Wrote inspect data to must-gather.
2026-04-20T12:59:32.962624050Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-20T12:59:33.022913891Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T12:59:33.184911948Z Gathering data for ns/openshift-network-console...
2026-04-20T12:59:33.194894256Z Wrote inspect data to must-gather.
2026-04-20T12:59:33.321595325Z INFO: Getting status from alertmanager-main-0
2026-04-20T12:59:33.482281720Z Wrote inspect data to must-gather.
2026-04-20T12:59:33.589623327Z Gathering data for ns/openshift-console-operator...
2026-04-20T12:59:33.723065133Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-20T12:59:33.725109510Z INFO: Fallback to identify the container image from release info
2026-04-20T12:59:33.933783949Z Gathering data for ns/openshift-console...
2026-04-20T12:59:34.226270982Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-20T12:59:34.345288132Z daemonset.apps/perf-node-gather-daemonset created
2026-04-20T12:59:34.347282955Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-20T12:59:34.471010854Z Waiting for performance profile collector pods to become ready: 1
2026-04-20T12:59:34.685481473Z Gathering data for ns/openshift-dns-operator...
2026-04-20T12:59:34.957244548Z Gathering data for ns/openshift-dns...
2026-04-20T12:59:35.483755126Z Gathering data for ns/openshift-image-registry...
2026-04-20T12:59:35.578666580Z Waiting for performance profile collector pods to become ready: 2
2026-04-20T12:59:35.910240846Z Gathering data for ns/openshift-ingress-operator...
2026-04-20T12:59:36.192745882Z Gathering data for ns/openshift-ingress...
2026-04-20T12:59:36.510016213Z Gathering data for ns/openshift-ingress-canary...
2026-04-20T12:59:36.565720392Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T12:59:36.699220367Z Waiting for performance profile collector pods to become ready: 3
2026-04-20T12:59:36.860179743Z Gathering data for ns/openshift-insights...
2026-04-20T12:59:37.812126704Z Waiting for performance profile collector pods to become ready: 4
2026-04-20T12:59:38.627323549Z Gathering data for ns/openshift-jobset-operator...
2026-04-20T12:59:38.935089420Z Waiting for performance profile collector pods to become ready: 5
2026-04-20T12:59:39.392028725Z Gathering data for ns/openshift-config...
2026-04-20T12:59:39.674681944Z Gathering data for ns/openshift-config-managed...
2026-04-20T12:59:40.059657863Z Waiting for performance profile collector pods to become ready: 6
2026-04-20T12:59:40.090852579Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-20T12:59:40.356544329Z Gathering data for ns/openshift-kube-apiserver...
2026-04-20T12:59:40.610486569Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-20T12:59:40.900294365Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-20T12:59:41.178137612Z Waiting for performance profile collector pods to become ready: 7
2026-04-20T12:59:41.181245226Z Gathering data for ns/openshift-kube-scheduler...
2026-04-20T12:59:41.427171039Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-20T12:59:41.570395185Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T12:59:41.757482718Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-20T12:59:42.124940632Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-20T12:59:42.286212417Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-20T12:59:42.474019662Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-20T12:59:42.549699348Z Collecting performance related data for node ip-10-0-137-91.ec2.internal
2026-04-20T12:59:42.556922353Z Collecting performance related data for node ip-10-0-135-187.ec2.internal
2026-04-20T12:59:42.564162011Z Collecting performance related data for node ip-10-0-131-55.ec2.internal
2026-04-20T12:59:43.033721375Z Gathering data for ns/openshift-multus...
2026-04-20T12:59:44.492722286Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-20T12:59:45.845299855Z Gathering data for ns/openshift-host-network...
2026-04-20T12:59:46.174242635Z Gathering data for ns/openshift-network-diagnostics...
2026-04-20T12:59:46.575268205Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T12:59:46.685445622Z Gathering data for ns/openshift-network-node-identity...
2026-04-20T12:59:47.144030546Z Gathering data for ns/openshift-network-operator...
2026-04-20T12:59:47.493441518Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-20T12:59:47.780908401Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-20T12:59:48.250496632Z Gathering data for ns/openshift-apiserver-operator...
2026-04-20T12:59:48.565299312Z Gathering data for ns/openshift-apiserver...
2026-04-20T12:59:48.902040496Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-20T12:59:49.145406662Z Gathering data for ns/openshift-controller-manager...
2026-04-20T12:59:49.388210164Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-20T12:59:49.996480830Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-20T12:59:50.264897966Z Gathering data for ns/openshift-service-ca-operator...
2026-04-20T12:59:50.338378362Z Collecting kubelet logs for node ip-10-0-137-91.ec2.internal
2026-04-20T12:59:50.340708111Z Collecting kubelet logs for node ip-10-0-135-187.ec2.internal
2026-04-20T12:59:50.342920625Z Collecting kubelet logs for node ip-10-0-131-55.ec2.internal
2026-04-20T12:59:50.595025630Z Gathering data for ns/openshift-service-ca...
2026-04-20T12:59:50.664231251Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-20T12:59:50.669775909Z INFO: Node performance data collection complete.
2026-04-20T12:59:50.965004160Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-20T12:59:51.582061156Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T12:59:51.804967054Z Wrote inspect data to must-gather.
2026-04-20T12:59:51.805012267Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T12:59:51.805012267Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-20T12:59:51.930481566Z error: the server doesn't have a resource type "clusters"
2026-04-20T12:59:52.071719058Z Caches written to disk