2026-04-16T14:47:29.529533451Z [disk usage checker] Started
2026-04-16T14:47:29.564217307Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T14:47:29.741986275Z Gathering data for ns/openshift-cluster-version...
2026-04-16T14:47:29.871557228Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T14:47:29.955943257Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T14:47:30.033157407Z Gathering data for ns/default...
2026-04-16T14:47:30.321897568Z Gathering data for ns/openshift...
2026-04-16T14:47:30.755948063Z Gathering data for ns/kube-system...
2026-04-16T14:47:31.446002217Z Gathering data for ns/openshift-etcd...
2026-04-16T14:47:31.743136322Z Wrote inspect data to must-gather.
2026-04-16T14:47:31.743169863Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T14:47:31.743169863Z     namespaces "assisted-installer" not found
2026-04-16T14:47:32.003855515Z Waiting on subprocesses to finish execution.
2026-04-16T14:47:32.008722875Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T14:47:32.015181663Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T14:47:32.015220779Z INFO: Gathering HAProxy config files
2026-04-16T14:47:32.019530037Z INFO: Collecting host service logs for crio
2026-04-16T14:47:32.019852438Z INFO: Collecting host service logs for kubelet
2026-04-16T14:47:32.020188058Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T14:47:32.020542908Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T14:47:32.020877311Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T14:47:32.021202904Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T14:47:32.021547126Z INFO: Collecting host service logs for NetworkManager
2026-04-16T14:47:32.021882248Z INFO: Collecting host service logs for openvswitch
2026-04-16T14:47:32.022209027Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T14:47:32.022602652Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T14:47:32.022920315Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T14:47:32.023290571Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T14:47:32.037849674Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T14:47:32.192396663Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T14:47:32.389493804Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T14:47:32.568251027Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-c2xv7
2026-04-16T14:47:32.568251027Z insights-runtime-extractor-8mgk9
2026-04-16T14:47:32.568251027Z insights-runtime-extractor-hq77c
2026-04-16T14:47:32.568251027Z insights-runtime-extractor-lmzjv
2026-04-16T14:47:32.841175383Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T14:47:32.882364099Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T14:47:32.923457793Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T14:47:32.961865031Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T14:47:32.961943400Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T14:47:33.063922757Z No resources found
2026-04-16T14:47:33.248882030Z No resources found in openshift-etcd namespace.
2026-04-16T14:47:33.301110057Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T14:47:33.345862833Z Wrote inspect data to must-gather.
2026-04-16T14:47:33.348628803Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T14:47:33.348628803Z See 'oc rsync -h' for help and examples
2026-04-16T14:47:33.354623485Z INFO: Worker host service log collection to complete.
2026-04-16T14:47:33.492250484Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T14:47:33.601450619Z INFO: 'previous-logs' folder not found on ip-10-0-128-129.ec2.internal, skipping...
2026-04-16T14:47:33.614421453Z ERROR: No running kube-apiserver pods found
2026-04-16T14:47:33.679461704Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T14:47:33.783536970Z Wrote inspect data to must-gather.
2026-04-16T14:47:33.889775468Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T14:47:33.960817536Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T14:47:33.996622446Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T14:47:34.021033713Z Gathering data for ns/openshift-monitoring...
2026-04-16T14:47:34.062639502Z No resources found
2026-04-16T14:47:34.259999846Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.272545493Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.327037944Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.329708023Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.344927902Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.388445565Z INFO: 'previous-logs' folder not found on ip-10-0-130-195.ec2.internal, skipping...
2026-04-16T14:47:34.461215960Z tar: Removing leading `/' from member names
2026-04-16T14:47:34.465451705Z INFO: HAProxy config collection complete.
2026-04-16T14:47:34.505953956Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T14:47:34.573876289Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T14:47:34.579979216Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T14:47:34.610243987Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.676073675Z INFO: 'previous-logs' folder not found on ip-10-0-136-109.ec2.internal, skipping...
2026-04-16T14:47:34.676176144Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T14:47:34.676192779Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T14:47:34.739273554Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.864085159Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.953958070Z Wrote inspect data to must-gather.
2026-04-16T14:47:34.982400199Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T14:47:35.103624099Z Wrote inspect data to must-gather.
2026-04-16T14:47:35.119817740Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T14:47:35.334036251Z Wrote inspect data to must-gather.
2026-04-16T14:47:35.403184929Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T14:47:35.403786185Z INFO: INTERCONNECT MODE
2026-04-16T14:47:35.403807792Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T14:47:35.411909170Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T14:47:35.571922145Z INFO: Gathering OVN_Northbound from ovnkube-node-2vdts...
2026-04-16T14:47:35.572163561Z INFO: Gathering OVN_Northbound from ovnkube-node-fkt9w...
2026-04-16T14:47:35.572381237Z INFO: Gathering OVN_Northbound from ovnkube-node-kftpl...
2026-04-16T14:47:35.572684255Z INFO: Gathering OVN_Southbound from ovnkube-node-2vdts...
2026-04-16T14:47:35.572918872Z INFO: Gathering OVN_Southbound from ovnkube-node-fkt9w...
2026-04-16T14:47:35.573185910Z INFO: Gathering OVN_Southbound from ovnkube-node-kftpl...
2026-04-16T14:47:35.575997907Z Wrote inspect data to must-gather.
2026-04-16T14:47:35.682119713Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T14:47:35.873530857Z tar: Removing leading `/' from member names
2026-04-16T14:47:35.876079270Z tar: Removing leading `/' from member names
2026-04-16T14:47:35.885019030Z tar: Removing leading `/' from member names
2026-04-16T14:47:35.897426965Z tar: Removing leading `/' from member names
2026-04-16T14:47:35.909542992Z tar: Removing leading `/' from member names
2026-04-16T14:47:35.917942484Z tar: Removing leading `/' from member names
2026-04-16T14:47:35.927665636Z Wrote inspect data to must-gather.
2026-04-16T14:47:36.045357263Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T14:47:36.093083460Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T14:47:36.240165428Z Wrote inspect data to must-gather.
2026-04-16T14:47:36.293459376Z Gathering data for ns/openshift-network-console...
2026-04-16T14:47:36.316855289Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T14:47:36.451901068Z INFO: Waiting for network log collection to complete ...
2026-04-16T14:47:36.452019174Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T14:47:36.452578470Z INFO: Copying ovnk databases complete.
2026-04-16T14:47:36.456805609Z 3.8M	must-gather/network_logs/ovnk_database_store
2026-04-16T14:47:36.462822321Z ovnk_database_store/
2026-04-16T14:47:36.462876321Z ovnk_database_store/ovnkube-node-2vdts_sbdb
2026-04-16T14:47:36.483775739Z ovnk_database_store/ovnkube-node-fkt9w_sbdb
2026-04-16T14:47:36.502407169Z ovnk_database_store/ovnkube-node-kftpl_nbdb
2026-04-16T14:47:36.514565152Z ovnk_database_store/ovnkube-node-kftpl_sbdb
2026-04-16T14:47:36.538551453Z ovnk_database_store/ovnkube-node-fkt9w_nbdb
2026-04-16T14:47:36.548507423Z ovnk_database_store/ovnkube-node-2vdts_nbdb
2026-04-16T14:47:36.554659249Z Wrote inspect data to must-gather.
2026-04-16T14:47:36.560937029Z INFO: Network log collection complete.
2026-04-16T14:47:36.742684224Z INFO: Getting status from alertmanager-main-0
2026-04-16T14:47:36.769849355Z Gathering data for ns/openshift-console-operator...
2026-04-16T14:47:36.848922192Z Wrote inspect data to must-gather.
2026-04-16T14:47:37.141125757Z Wrote inspect data to must-gather.
2026-04-16T14:47:37.195695080Z Gathering data for ns/openshift-console...
2026-04-16T14:47:37.402704083Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T14:47:37.405913309Z INFO: Fallback to identify the container image from release info
2026-04-16T14:47:37.627250234Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T14:47:37.912037940Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T14:47:37.963590609Z Gathering data for ns/openshift-dns-operator...
2026-04-16T14:47:38.054178345Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T14:47:38.182270645Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T14:47:38.263149318Z Gathering data for ns/openshift-dns...
2026-04-16T14:47:38.837425283Z Gathering data for ns/openshift-image-registry...
2026-04-16T14:47:39.296669213Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T14:47:39.321044331Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T14:47:39.578898397Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T14:47:39.602631936Z Gathering data for ns/openshift-ingress...
2026-04-16T14:47:39.920355452Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T14:47:40.317729825Z Gathering data for ns/openshift-insights...
2026-04-16T14:47:40.437150895Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T14:47:41.559451407Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T14:47:42.427777805Z Gathering data for ns/kserve...
2026-04-16T14:47:42.683663624Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T14:47:43.818079525Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T14:47:44.044081261Z Gathering data for ns/openshift-config...
2026-04-16T14:47:44.277579085Z Gathering data for ns/openshift-config-managed...
2026-04-16T14:47:44.583883932Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T14:47:44.667406571Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T14:47:44.938339704Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T14:47:44.954134249Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T14:47:45.187777772Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T14:47:45.443623412Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T14:47:45.701014746Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T14:47:45.951035378Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T14:47:46.090014732Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T14:47:46.269190560Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T14:47:46.375398933Z Collecting performance related data for node ip-10-0-130-195.ec2.internal
2026-04-16T14:47:46.383127055Z Collecting performance related data for node ip-10-0-128-129.ec2.internal
2026-04-16T14:47:46.391230425Z Collecting performance related data for node ip-10-0-136-109.ec2.internal
2026-04-16T14:47:46.624517381Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T14:47:46.988606981Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T14:47:47.582430068Z Gathering data for ns/openshift-multus...
2026-04-16T14:47:48.962461772Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T14:47:49.589290493Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T14:47:50.484735712Z Gathering data for ns/openshift-host-network...
2026-04-16T14:47:50.818546305Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T14:47:51.336002274Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T14:47:51.760954312Z Gathering data for ns/openshift-network-operator...
2026-04-16T14:47:52.131777260Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T14:47:52.400983013Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T14:47:52.834221856Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T14:47:53.179588588Z Gathering data for ns/openshift-apiserver...
2026-04-16T14:47:53.632364879Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T14:47:53.888976616Z Gathering data for ns/openshift-controller-manager...
2026-04-16T14:47:54.137375599Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T14:47:54.597098636Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T14:47:54.744871116Z Collecting kubelet logs for node ip-10-0-130-195.ec2.internal
2026-04-16T14:47:54.747063341Z Collecting kubelet logs for node ip-10-0-128-129.ec2.internal
2026-04-16T14:47:54.748769832Z Collecting kubelet logs for node ip-10-0-136-109.ec2.internal
2026-04-16T14:47:54.779422449Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T14:47:55.125650773Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T14:47:55.128431873Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T14:47:55.130649044Z INFO: Node performance data collection complete.
2026-04-16T14:47:55.442564543Z Gathering data for ns/openshift-service-ca...
2026-04-16T14:47:55.775714641Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T14:47:56.606814352Z Wrote inspect data to must-gather.
2026-04-16T14:47:56.606859568Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T14:47:56.606859568Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T14:47:56.740200763Z error: the server doesn't have a resource type "clusters"
2026-04-16T14:47:57.091257294Z Caches written to disk