2026-04-16T14:52:24.254938668Z [disk usage checker] Started
2026-04-16T14:52:24.285507089Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:24.490613501Z Gathering data for ns/openshift-cluster-version...
2026-04-16T14:52:24.656424250Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T14:52:24.763446873Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T14:52:24.873129456Z Gathering data for ns/default...
2026-04-16T14:52:25.272166560Z Gathering data for ns/openshift...
2026-04-16T14:52:25.783667925Z Gathering data for ns/kube-system...
2026-04-16T14:52:26.603729786Z Gathering data for ns/openshift-etcd...
2026-04-16T14:52:26.824838533Z Waiting on subprocesses to finish execution.
2026-04-16T14:52:26.828550257Z INFO: Gathering HAProxy config files
2026-04-16T14:52:26.838195578Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T14:52:26.845875259Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T14:52:26.848259179Z INFO: Collecting host service logs for crio
2026-04-16T14:52:26.848577083Z INFO: Collecting host service logs for kubelet
2026-04-16T14:52:26.848895419Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T14:52:26.849213595Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T14:52:26.849613521Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T14:52:26.849840704Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T14:52:26.850194598Z INFO: Collecting host service logs for NetworkManager
2026-04-16T14:52:26.850508735Z INFO: Collecting host service logs for openvswitch
2026-04-16T14:52:26.850867424Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T14:52:26.851183619Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T14:52:26.851488863Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T14:52:26.851874785Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T14:52:26.864942706Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T14:52:26.960741475Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T14:52:27.738835043Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-cnkw9
2026-04-16T14:52:27.738886121Z insights-runtime-extractor-45bjl
2026-04-16T14:52:27.738906905Z insights-runtime-extractor-6w8wv
2026-04-16T14:52:27.738926590Z insights-runtime-extractor-8fb7x
2026-04-16T14:52:27.772032513Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T14:52:27.885199090Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T14:52:27.937739131Z No resources found in openshift-etcd namespace.
2026-04-16T14:52:27.966838770Z Wrote inspect data to must-gather.
2026-04-16T14:52:28.138269301Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T14:52:28.194848216Z Wrote inspect data to must-gather.
2026-04-16T14:52:28.194973186Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T14:52:28.194973186Z     namespaces "assisted-installer" not found
2026-04-16T14:52:28.277761786Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T14:52:28.315801745Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T14:52:28.316110144Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T14:52:28.353308705Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T14:52:28.353308705Z See 'oc rsync -h' for help and examples
2026-04-16T14:52:28.381579991Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T14:52:28.442822636Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T14:52:28.553692363Z INFO: Worker host service log collection to complete.
2026-04-16T14:52:28.597042201Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T14:52:28.616500660Z Wrote inspect data to must-gather.
2026-04-16T14:52:28.635863290Z No resources found
2026-04-16T14:52:28.683817173Z ERROR: No running kube-apiserver pods found
2026-04-16T14:52:28.725810542Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T14:52:29.009985213Z INFO: HAProxy config collection complete.
2026-04-16T14:52:29.144566867Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T14:52:29.457888446Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:29.538342934Z Wrote inspect data to must-gather.
2026-04-16T14:52:29.574358588Z INFO: 'previous-logs' folder not found on ip-10-0-128-173.ec2.internal, skipping...
2026-04-16T14:52:29.662189115Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T14:52:29.693381525Z No resources found
2026-04-16T14:52:29.754482581Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T14:52:29.815742001Z Wrote inspect data to must-gather.
2026-04-16T14:52:29.898742743Z Gathering data for ns/kuadrant-system...
2026-04-16T14:52:29.933401302Z Wrote inspect data to must-gather.
2026-04-16T14:52:30.041792172Z Wrote inspect data to must-gather.
2026-04-16T14:52:30.050949067Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T14:52:30.185767710Z INFO: 'previous-logs' folder not found on ip-10-0-140-144.ec2.internal, skipping...
2026-04-16T14:52:30.227296670Z Wrote inspect data to must-gather.
2026-04-16T14:52:30.309604453Z Wrote inspect data to must-gather.
2026-04-16T14:52:30.542007440Z INFO: 'previous-logs' folder not found on ip-10-0-141-239.ec2.internal, skipping...
2026-04-16T14:52:30.542043025Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T14:52:30.542043025Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T14:52:30.555780445Z Wrote inspect data to must-gather.
2026-04-16T14:52:30.563689403Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T14:52:30.715382259Z Wrote inspect data to must-gather.
2026-04-16T14:52:30.802082508Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T14:52:30.867030887Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T14:52:31.071266250Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T14:52:31.071463377Z Wrote inspect data to must-gather.
2026-04-16T14:52:31.126648852Z Wrote inspect data to must-gather.
2026-04-16T14:52:31.339414510Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T14:52:31.422184783Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T14:52:31.452249720Z INFO: INTERCONNECT MODE
2026-04-16T14:52:31.452345190Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T14:52:31.529198725Z Wrote inspect data to must-gather.
2026-04-16T14:52:31.655972699Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T14:52:31.699890102Z Gathering data for ns/openshift-monitoring...
2026-04-16T14:52:31.719479570Z INFO: Gathering OVN_Northbound from ovnkube-node-8wdkz...
2026-04-16T14:52:31.719750287Z INFO: Gathering OVN_Northbound from ovnkube-node-p7qpv...
2026-04-16T14:52:31.719970015Z INFO: Gathering OVN_Northbound from ovnkube-node-vclsl...
2026-04-16T14:52:31.720304134Z INFO: Gathering OVN_Southbound from ovnkube-node-8wdkz...
2026-04-16T14:52:31.720599189Z INFO: Gathering OVN_Southbound from ovnkube-node-p7qpv...
2026-04-16T14:52:31.720901631Z INFO: Gathering OVN_Southbound from ovnkube-node-vclsl...
2026-04-16T14:52:31.927802968Z Wrote inspect data to must-gather.
2026-04-16T14:52:32.003308409Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T14:52:32.020606651Z tar: Removing leading `/' from member names
2026-04-16T14:52:32.076623306Z tar: Removing leading `/' from member names
2026-04-16T14:52:32.140943851Z tar: Removing leading `/' from member names
2026-04-16T14:52:32.200666263Z tar: Removing leading `/' from member names
2026-04-16T14:52:32.228907958Z tar: Removing leading `/' from member names
2026-04-16T14:52:32.253583161Z tar: Removing leading `/' from member names
2026-04-16T14:52:32.397755368Z Wrote inspect data to must-gather.
2026-04-16T14:52:32.594493181Z INFO: Getting status from alertmanager-main-0
2026-04-16T14:52:32.801051009Z INFO: Waiting for network log collection to complete ...
2026-04-16T14:52:32.801097628Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T14:52:32.801775421Z INFO: Copying ovnk databases complete.
2026-04-16T14:52:32.803074295Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-16T14:52:32.803502029Z Wrote inspect data to must-gather.
2026-04-16T14:52:32.805524839Z ovnk_database_store/
2026-04-16T14:52:32.805524839Z ovnk_database_store/ovnkube-node-p7qpv_sbdb
2026-04-16T14:52:32.834432541Z ovnk_database_store/ovnkube-node-p7qpv_nbdb
2026-04-16T14:52:32.847222851Z ovnk_database_store/ovnkube-node-8wdkz_sbdb
2026-04-16T14:52:32.877284786Z ovnk_database_store/ovnkube-node-vclsl_nbdb
2026-04-16T14:52:32.887116365Z ovnk_database_store/ovnkube-node-vclsl_sbdb
2026-04-16T14:52:32.907658085Z ovnk_database_store/ovnkube-node-8wdkz_nbdb
2026-04-16T14:52:32.920831319Z INFO: Network log collection complete.
2026-04-16T14:52:33.120228879Z Wrote inspect data to must-gather.
2026-04-16T14:52:33.436484761Z Wrote inspect data to must-gather.
2026-04-16T14:52:33.704936671Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T14:52:33.708577317Z INFO: Fallback to identify the container image from release info
2026-04-16T14:52:34.250772957Z Gathering data for ns/openshift-network-console...
2026-04-16T14:52:34.438491076Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T14:52:34.546947880Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:34.587718019Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T14:52:34.750289994Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T14:52:34.856806261Z Gathering data for ns/openshift-console-operator...
2026-04-16T14:52:35.394159838Z Gathering data for ns/openshift-console...
2026-04-16T14:52:35.903627638Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T14:52:36.004941829Z Wrote inspect data to must-gather.
2026-04-16T14:52:36.011077829Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T14:52:36.435085570Z Gathering data for ns/openshift-dns-operator...
2026-04-16T14:52:36.778482023Z Gathering data for ns/openshift-dns...
2026-04-16T14:52:37.014772751Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T14:52:37.448254675Z Gathering data for ns/openshift-image-registry...
2026-04-16T14:52:37.988130559Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T14:52:38.149704850Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T14:52:38.410354318Z Gathering data for ns/openshift-ingress...
2026-04-16T14:52:39.152069449Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T14:52:39.261684195Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T14:52:39.552649847Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:39.669492331Z Gathering data for ns/openshift-insights...
2026-04-16T14:52:40.400373045Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T14:52:41.530258043Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T14:52:41.796184138Z Gathering data for ns/opendatahub...
2026-04-16T14:52:42.667497615Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T14:52:42.947758309Z Collecting performance related data for node ip-10-0-128-173.ec2.internal
2026-04-16T14:52:42.955082568Z Collecting performance related data for node ip-10-0-141-239.ec2.internal
2026-04-16T14:52:42.963313103Z Collecting performance related data for node ip-10-0-140-144.ec2.internal
2026-04-16T14:52:43.435894839Z Gathering data for ns/openshift-lws-operator...
2026-04-16T14:52:44.558987372Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:45.076316081Z Gathering data for ns/openshift-config...
2026-04-16T14:52:45.486690013Z Gathering data for ns/openshift-config-managed...
2026-04-16T14:52:46.086862452Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T14:52:46.478276126Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T14:52:46.819785960Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T14:52:47.227773938Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T14:52:47.623969365Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T14:52:47.977015232Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T14:52:48.387759792Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T14:52:48.800098844Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T14:52:49.202950570Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T14:52:49.566843476Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:49.856279349Z Gathering data for ns/openshift-multus...
2026-04-16T14:52:51.523450276Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T14:52:51.526296150Z Collecting kubelet logs for node ip-10-0-128-173.ec2.internal
2026-04-16T14:52:51.528185253Z Collecting kubelet logs for node ip-10-0-141-239.ec2.internal
2026-04-16T14:52:51.529934823Z Collecting kubelet logs for node ip-10-0-140-144.ec2.internal
2026-04-16T14:52:51.870250898Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T14:52:51.875661631Z INFO: Node performance data collection complete.
2026-04-16T14:52:53.216322083Z Gathering data for ns/openshift-host-network...
2026-04-16T14:52:53.621066740Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T14:52:54.246131368Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T14:52:54.576578933Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:54.767266464Z Gathering data for ns/openshift-network-operator...
2026-04-16T14:52:55.219323397Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T14:52:55.573169849Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T14:52:56.102816333Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T14:52:56.469339940Z Gathering data for ns/openshift-apiserver...
2026-04-16T14:52:56.956839553Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T14:52:57.363955058Z Gathering data for ns/openshift-controller-manager...
2026-04-16T14:52:57.737483187Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T14:52:58.460818555Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T14:52:58.852524146Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T14:52:59.260999394Z Gathering data for ns/openshift-service-ca...
2026-04-16T14:52:59.585813823Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-04-16T14:52:59.715002812Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T14:53:00.838782537Z Wrote inspect data to must-gather.
2026-04-16T14:53:00.838835538Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T14:53:00.838835538Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T14:53:00.976056241Z error: the server doesn't have a resource type "clusters"
2026-04-16T14:53:01.160751114Z Caches written to disk