2026-04-24T23:33:35.628516111Z [disk usage checker] Started
2026-04-24T23:33:35.632437711Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70
2026-04-24T23:33:35.947059105Z Gathering data for ns/openshift-cluster-version...
2026-04-24T23:33:36.078046662Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T23:33:36.163900041Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T23:33:36.242150217Z Gathering data for ns/default...
2026-04-24T23:33:36.584610284Z Gathering data for ns/openshift...
2026-04-24T23:33:36.982354734Z Gathering data for ns/kube-system...
2026-04-24T23:33:37.772854908Z Gathering data for ns/openshift-etcd...
2026-04-24T23:33:38.072295055Z Wrote inspect data to must-gather.
2026-04-24T23:33:38.072326455Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T23:33:38.072326455Z     namespaces "assisted-installer" not found
2026-04-24T23:33:38.163198307Z Waiting on subprocesses to finish execution.
2026-04-24T23:33:38.168899072Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T23:33:38.171616777Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T23:33:38.177482403Z INFO: Gathering HAProxy config files
2026-04-24T23:33:38.183130963Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T23:33:38.184279849Z INFO: Collecting host service logs for crio
2026-04-24T23:33:38.184625631Z INFO: Collecting host service logs for kubelet
2026-04-24T23:33:38.184968170Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T23:33:38.185294703Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T23:33:38.185649756Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T23:33:38.185977034Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T23:33:38.186303097Z INFO: Collecting host service logs for NetworkManager
2026-04-24T23:33:38.186659600Z INFO: Collecting host service logs for openvswitch
2026-04-24T23:33:38.186990863Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T23:33:38.187330826Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T23:33:38.187704793Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T23:33:38.188254660Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T23:33:38.244651982Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T23:33:38.780164679Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T23:33:38.780200876Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T23:33:38.831916383Z No resources found in openshift-etcd namespace.
2026-04-24T23:33:39.057896433Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T23:33:39.215676912Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-ddhcv
2026-04-24T23:33:39.215723879Z insights-runtime-extractor-cr5q4
2026-04-24T23:33:39.215743468Z insights-runtime-extractor-lp9hc
2026-04-24T23:33:39.215762391Z insights-runtime-extractor-n6js7
2026-04-24T23:33:39.280493825Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T23:33:39.288568198Z No resources found
2026-04-24T23:33:39.313690843Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T23:33:39.407945008Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T23:33:39.412274537Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T23:33:39.430985816Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T23:33:39.495175899Z INFO: Worker host service log collection to complete.
2026-04-24T23:33:39.526326270Z ERROR: No running kube-apiserver pods found
2026-04-24T23:33:39.592033068Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T23:33:39.592033068Z See 'oc rsync -h' for help and examples
2026-04-24T23:33:39.670157285Z Wrote inspect data to must-gather.
2026-04-24T23:33:39.677990602Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T23:33:39.789531961Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.011129396Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T23:33:40.036544913Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.211771933Z Gathering data for ns/openshift-monitoring...
2026-04-24T23:33:40.250802538Z No resources found
2026-04-24T23:33:40.268970754Z tar: Removing leading `/' from member names
2026-04-24T23:33:40.281225295Z INFO: HAProxy config collection complete.
2026-04-24T23:33:40.315293339Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T23:33:40.377614945Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T23:33:40.383323599Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.392625467Z INFO: 'previous-logs' folder not found on ip-10-0-134-116.ec2.internal, skipping...
2026-04-24T23:33:40.476981309Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.512280416Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T23:33:40.543892709Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.595889221Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.639873467Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70
2026-04-24T23:33:40.682487781Z INFO: 'previous-logs' folder not found on ip-10-0-136-66.ec2.internal, skipping...
2026-04-24T23:33:40.746149340Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.840532781Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T23:33:40.854829009Z Wrote inspect data to must-gather.
2026-04-24T23:33:40.960368068Z INFO: 'previous-logs' folder not found on ip-10-0-142-173.ec2.internal, skipping...
2026-04-24T23:33:40.960401424Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T23:33:40.960412448Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T23:33:40.991410012Z Wrote inspect data to must-gather.
2026-04-24T23:33:41.114854289Z Wrote inspect data to must-gather.
2026-04-24T23:33:41.148448332Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T23:33:41.248570540Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T23:33:41.355998099Z Wrote inspect data to must-gather.
2026-04-24T23:33:41.362703680Z Wrote inspect data to must-gather.
2026-04-24T23:33:41.595406716Z Wrote inspect data to must-gather.
2026-04-24T23:33:41.681495691Z INFO: INTERCONNECT MODE
2026-04-24T23:33:41.681526235Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T23:33:41.690852264Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T23:33:41.716329329Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T23:33:41.817371938Z Wrote inspect data to must-gather.
2026-04-24T23:33:41.853982624Z INFO: Gathering OVN_Northbound from ovnkube-node-ckgcl...
2026-04-24T23:33:41.854302032Z INFO: Gathering OVN_Northbound from ovnkube-node-t72f2...
2026-04-24T23:33:41.854552663Z INFO: Gathering OVN_Northbound from ovnkube-node-tbc58...
2026-04-24T23:33:41.854840002Z INFO: Gathering OVN_Southbound from ovnkube-node-ckgcl...
2026-04-24T23:33:41.855103257Z INFO: Gathering OVN_Southbound from ovnkube-node-t72f2...
2026-04-24T23:33:41.855433392Z INFO: Gathering OVN_Southbound from ovnkube-node-tbc58...
2026-04-24T23:33:42.003184799Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T23:33:42.144191217Z tar: Removing leading `/' from member names
2026-04-24T23:33:42.165516236Z tar: Removing leading `/' from member names
2026-04-24T23:33:42.197519772Z Wrote inspect data to must-gather.
2026-04-24T23:33:42.197519772Z tar: Removing leading `/' from member names
2026-04-24T23:33:42.205045268Z tar: Removing leading `/' from member names
2026-04-24T23:33:42.207952710Z tar: Removing leading `/' from member names
2026-04-24T23:33:42.264443846Z tar: Removing leading `/' from member names
2026-04-24T23:33:42.331900960Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T23:33:42.438316583Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T23:33:42.540568552Z Wrote inspect data to must-gather.
2026-04-24T23:33:42.638797527Z Gathering data for ns/openshift-network-console...
2026-04-24T23:33:42.640745411Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T23:33:42.731940748Z INFO: Waiting for network log collection to complete ...
2026-04-24T23:33:42.732036569Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T23:33:42.732488993Z INFO: Copying ovnk databases complete.
2026-04-24T23:33:42.734619601Z 4.4M	must-gather/network_logs/ovnk_database_store
2026-04-24T23:33:42.737544580Z ovnk_database_store/
2026-04-24T23:33:42.737544580Z ovnk_database_store/ovnkube-node-t72f2_nbdb
2026-04-24T23:33:42.755874828Z ovnk_database_store/ovnkube-node-t72f2_sbdb
2026-04-24T23:33:42.785119907Z ovnk_database_store/ovnkube-node-ckgcl_nbdb
2026-04-24T23:33:42.795070011Z ovnk_database_store/ovnkube-node-tbc58_nbdb
2026-04-24T23:33:42.804681999Z ovnk_database_store/ovnkube-node-tbc58_sbdb
2026-04-24T23:33:42.826445091Z ovnk_database_store/ovnkube-node-ckgcl_sbdb
2026-04-24T23:33:42.849951013Z INFO: Network log collection complete.
2026-04-24T23:33:42.850682739Z Wrote inspect data to must-gather.
2026-04-24T23:33:43.079280639Z INFO: Getting status from alertmanager-main-0
2026-04-24T23:33:43.110621882Z Gathering data for ns/openshift-console-operator...
2026-04-24T23:33:43.180840319Z Wrote inspect data to must-gather.
2026-04-24T23:33:43.439531286Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T23:33:43.443517403Z INFO: Fallback to identify the container image from release info
2026-04-24T23:33:43.495113100Z Gathering data for ns/openshift-console...
2026-04-24T23:33:43.915236563Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T23:33:43.944207669Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T23:33:44.078510654Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T23:33:44.219279536Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T23:33:44.284038171Z Gathering data for ns/openshift-dns-operator...
2026-04-24T23:33:44.612557091Z Gathering data for ns/openshift-dns...
2026-04-24T23:33:45.166727181Z Gathering data for ns/openshift-image-registry...
2026-04-24T23:33:45.348212900Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T23:33:45.625651652Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T23:33:45.646573150Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70
2026-04-24T23:33:45.934900821Z Gathering data for ns/openshift-ingress...
2026-04-24T23:33:46.261959101Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T23:33:46.478104415Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T23:33:46.632119140Z Gathering data for ns/openshift-insights...
2026-04-24T23:33:47.600438914Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T23:33:48.716539323Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T23:33:48.758593114Z Gathering data for ns/kserve...
2026-04-24T23:33:49.856396811Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T23:33:50.499768815Z Gathering data for ns/openshift-config...
2026-04-24T23:33:50.651448925Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70
2026-04-24T23:33:50.790722807Z Gathering data for ns/openshift-config-managed...
2026-04-24T23:33:50.975100526Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T23:33:51.199614658Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T23:33:51.471220191Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T23:33:51.705994533Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T23:33:51.995187772Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T23:33:52.110005614Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T23:33:52.329315398Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T23:33:52.382907368Z Collecting performance related data for node ip-10-0-136-66.ec2.internal
2026-04-24T23:33:52.391745122Z Collecting performance related data for node ip-10-0-134-116.ec2.internal
2026-04-24T23:33:52.402428998Z Collecting performance related data for node ip-10-0-142-173.ec2.internal
2026-04-24T23:33:52.628825379Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T23:33:52.958022013Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T23:33:53.340908946Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T23:33:53.718944372Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T23:33:54.273689348Z Gathering data for ns/openshift-multus...
2026-04-24T23:33:55.656156751Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70
2026-04-24T23:33:55.843630868Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T23:33:57.359012686Z Gathering data for ns/openshift-host-network...
2026-04-24T23:33:57.683907960Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T23:33:58.198013142Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T23:33:58.627591259Z Gathering data for ns/openshift-network-operator...
2026-04-24T23:33:58.987600206Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T23:33:59.277184794Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T23:33:59.717506621Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T23:34:00.040852751Z Gathering data for ns/openshift-apiserver...
2026-04-24T23:34:00.454113606Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T23:34:00.661850147Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70
2026-04-24T23:34:00.704411500Z Gathering data for ns/openshift-controller-manager...
2026-04-24T23:34:00.821790488Z Collecting kubelet logs for node ip-10-0-136-66.ec2.internal
2026-04-24T23:34:00.823659960Z Collecting kubelet logs for node ip-10-0-134-116.ec2.internal
2026-04-24T23:34:00.825960221Z Collecting kubelet logs for node ip-10-0-142-173.ec2.internal
2026-04-24T23:34:01.015426221Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T23:34:01.278591051Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T23:34:01.285214862Z INFO: Node performance data collection complete.
2026-04-24T23:34:01.691829022Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T23:34:02.000013559Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T23:34:02.282676222Z Gathering data for ns/openshift-service-ca...
2026-04-24T23:34:02.636151258Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T23:34:03.482367919Z Wrote inspect data to must-gather.
2026-04-24T23:34:03.482405156Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T23:34:03.482405156Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T23:34:03.612601227Z error: the server doesn't have a resource type "clusters"
2026-04-24T23:34:03.783781105Z Caches written to disk