2026-07-20T20:20:08.945602576Z [disk usage checker] Started
2026-07-20T20:20:08.949191523Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T20:20:09.138543895Z Gathering data for ns/openshift-cluster-version...
2026-07-20T20:20:09.276438566Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T20:20:09.354217070Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T20:20:09.431721485Z Gathering data for ns/default...
2026-07-20T20:20:09.709445653Z Gathering data for ns/openshift...
2026-07-20T20:20:10.104811241Z Gathering data for ns/kube-system...
2026-07-20T20:20:10.729314095Z Gathering data for ns/openshift-etcd...
2026-07-20T20:20:11.013916231Z Wrote inspect data to must-gather.
2026-07-20T20:20:11.013945416Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T20:20:11.013945416Z     namespaces "assisted-installer" not found
2026-07-20T20:20:11.475742072Z Waiting on subprocesses to finish execution.
2026-07-20T20:20:11.484102291Z INFO: Gathering HAProxy config files
2026-07-20T20:20:11.485126495Z INFO: Collecting host service logs for crio
2026-07-20T20:20:11.485391034Z INFO: Collecting host service logs for kubelet
2026-07-20T20:20:11.485984092Z INFO: Collecting host service logs for rpm-ostreed
2026-07-20T20:20:11.486278725Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-20T20:20:11.486331165Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-20T20:20:11.486636523Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-20T20:20:11.487130782Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-20T20:20:11.487438433Z INFO: Collecting host service logs for NetworkManager
2026-07-20T20:20:11.487736447Z INFO: Collecting host service logs for openvswitch
2026-07-20T20:20:11.488085573Z INFO: Collecting host service logs for ovs-configuration
2026-07-20T20:20:11.488379758Z INFO: Collecting host service logs for ovsdb-server
2026-07-20T20:20:11.488692813Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-20T20:20:11.489070615Z INFO: Waiting for worker host service log collection to complete ...
2026-07-20T20:20:11.489213417Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-20T20:20:11.489765720Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-20T20:20:11.499118284Z Executing Istio gather script
2026-07-20T20:20:11.645826066Z INFO: Waiting for node performance related collection to complete ...
2026-07-20T20:20:12.067330441Z INFO: "metallb-operator" not detected. Skipping.
2026-07-20T20:20:12.193199399Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-20T20:20:12.336185814Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-20T20:20:12.495091091Z Wrote inspect data to must-gather.
2026-07-20T20:20:12.612216170Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-20T20:20:12.675615591Z No resources found in openshift-etcd namespace.
2026-07-20T20:20:12.678048377Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-20T20:20:12.688648688Z No resources found
2026-07-20T20:20:12.801616127Z Wrote inspect data to must-gather.
2026-07-20T20:20:12.806976040Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-20T20:20:12.807013657Z INFO: on-disk MachineConfig config collection complete.
2026-07-20T20:20:12.873928123Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-20T20:20:12.882969303Z INFO: Worker host service log collection to complete.
2026-07-20T20:20:12.886634115Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-dqdt2
2026-07-20T20:20:12.886669461Z insights-runtime-extractor-4fsbm
2026-07-20T20:20:12.886691650Z insights-runtime-extractor-knspp
2026-07-20T20:20:12.886711209Z insights-runtime-extractor-kvntk
2026-07-20T20:20:13.020505874Z ERROR: No running kube-apiserver pods found
2026-07-20T20:20:13.033005668Z error: the server doesn't have a resource type "performanceprofile"
2026-07-20T20:20:13.182249274Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-20T20:20:13.182249274Z See 'oc rsync -h' for help and examples
2026-07-20T20:20:13.261080630Z tar: Removing leading `/' from member names
2026-07-20T20:20:13.265668874Z INFO: HAProxy config collection complete.
2026-07-20T20:20:13.325135215Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-20T20:20:13.356514769Z INFO: 'previous-logs' folder not found on ip-10-0-129-241.ec2.internal, skipping...
2026-07-20T20:20:13.364276112Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T20:20:13.508625583Z Wrote inspect data to must-gather.
2026-07-20T20:20:13.685310414Z Wrote inspect data to must-gather.
2026-07-20T20:20:13.918583829Z Wrote inspect data to must-gather.
2026-07-20T20:20:13.949437669Z No resources found
2026-07-20T20:20:13.989550817Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-20T20:20:14.009039546Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T20:20:14.011299935Z Gathering data for ns/openshift-monitoring...
2026-07-20T20:20:14.077556689Z INFO: 'previous-logs' folder not found on ip-10-0-139-133.ec2.internal, skipping...
2026-07-20T20:20:14.100389221Z Wrote inspect data to must-gather.
2026-07-20T20:20:14.124139388Z error: the server doesn't have a resource type "machineconfigs"
2026-07-20T20:20:14.154166719Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-20T20:20:14.287265679Z Wrote inspect data to must-gather.
2026-07-20T20:20:14.331204536Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-20T20:20:14.378637931Z INFO: Getting rules from prometheus-k8s-0
2026-07-20T20:20:14.383611754Z Wrote inspect data to must-gather.
2026-07-20T20:20:14.400662351Z INFO: 'previous-logs' folder not found on ip-10-0-142-28.ec2.internal, skipping...
2026-07-20T20:20:14.400735984Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-20T20:20:14.400785112Z INFO: Machine Config Daemon termination log collection complete.
2026-07-20T20:20:14.554674311Z Wrote inspect data to must-gather.
2026-07-20T20:20:14.625675659Z Wrote inspect data to must-gather.
2026-07-20T20:20:14.710657499Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-20T20:20:14.768962700Z INFO: Getting status/config from prometheus-k8s-0
2026-07-20T20:20:14.880260726Z Wrote inspect data to must-gather.
2026-07-20T20:20:14.923197863Z Wrote inspect data to must-gather.
2026-07-20T20:20:15.155005862Z Wrote inspect data to must-gather.
2026-07-20T20:20:15.170070382Z INFO: INTERCONNECT MODE
2026-07-20T20:20:15.170153433Z INFO: Gathering ovn-kubernetes DBs
2026-07-20T20:20:15.173583886Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-20T20:20:15.316465711Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T20:20:15.339403623Z INFO: Gathering OVN_Northbound from ovnkube-node-9l6rb...
2026-07-20T20:20:15.339637708Z INFO: Gathering OVN_Northbound from ovnkube-node-b6w2f...
2026-07-20T20:20:15.339907741Z INFO: Gathering OVN_Northbound from ovnkube-node-w96ft...
2026-07-20T20:20:15.340186323Z INFO: Gathering OVN_Southbound from ovnkube-node-9l6rb...
2026-07-20T20:20:15.340631348Z INFO: Gathering OVN_Southbound from ovnkube-node-b6w2f...
2026-07-20T20:20:15.340956318Z INFO: Gathering OVN_Southbound from ovnkube-node-w96ft...
2026-07-20T20:20:15.405717208Z Wrote inspect data to must-gather.
2026-07-20T20:20:15.472815706Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-20T20:20:15.535538659Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-20T20:20:15.663433833Z tar: Removing leading `/' from member names
2026-07-20T20:20:15.706084527Z tar: Removing leading `/' from member names
2026-07-20T20:20:15.749725006Z tar: Removing leading `/' from member names
2026-07-20T20:20:15.749855312Z tar: Removing leading `/' from member names
2026-07-20T20:20:15.750180732Z tar: Removing leading `/' from member names
2026-07-20T20:20:15.773948102Z tar: Removing leading `/' from member names
2026-07-20T20:20:15.830702335Z Wrote inspect data to must-gather.
2026-07-20T20:20:15.850557523Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-20T20:20:15.861084145Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:16.135030259Z W0720 20:20:16.134981    1401 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-20T20:20:16.142680672Z Wrote inspect data to must-gather.
2026-07-20T20:20:16.145022634Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:16.148651767Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-20T20:20:16.156545242Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-20T20:20:16.219878456Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T20:20:16.273727798Z INFO: Waiting for network log collection to complete ...
2026-07-20T20:20:16.273781731Z INFO: Waiting for ovnk database copies to complete ...
2026-07-20T20:20:16.274320898Z INFO: Copying ovnk databases complete.
2026-07-20T20:20:16.280617385Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-20T20:20:16.283287016Z ovnk_database_store/
2026-07-20T20:20:16.283336772Z ovnk_database_store/ovnkube-node-9l6rb_nbdb
2026-07-20T20:20:16.300013640Z ovnk_database_store/ovnkube-node-9l6rb_sbdb
2026-07-20T20:20:16.319652546Z ovnk_database_store/ovnkube-node-w96ft_sbdb
2026-07-20T20:20:16.336079146Z ovnk_database_store/ovnkube-node-b6w2f_nbdb
2026-07-20T20:20:16.342925040Z ovnk_database_store/ovnkube-node-b6w2f_sbdb
2026-07-20T20:20:16.360031792Z ovnk_database_store/ovnkube-node-w96ft_nbdb
2026-07-20T20:20:16.369563558Z INFO: Network log collection complete.
2026-07-20T20:20:16.396390777Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:16.440521151Z Gathering data for ns/openshift-network-console...
2026-07-20T20:20:16.468295293Z Wrote inspect data to must-gather.
2026-07-20T20:20:16.576216789Z INFO: Getting status from alertmanager-main-0
2026-07-20T20:20:16.622452669Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:16.625777391Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-20T20:20:16.796664426Z Wrote inspect data to must-gather.
2026-07-20T20:20:16.842902437Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:16.883567120Z Gathering data for ns/openshift-console-operator...
2026-07-20T20:20:17.066625704Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:17.069525918Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-20T20:20:17.075959627Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-20T20:20:17.077735541Z INFO: Fallback to identify the container image from release info
2026-07-20T20:20:17.270023080Z Gathering data for ns/openshift-console...
2026-07-20T20:20:17.288581580Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:17.479206712Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:17.481766813Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-20T20:20:17.660481050Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:17.709094665Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-20T20:20:17.774040012Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-20T20:20:17.844426630Z daemonset.apps/perf-node-gather-daemonset created
2026-07-20T20:20:17.887363098Z Wrote inspect data to /must-gather/istio.
2026-07-20T20:20:17.988213123Z Waiting for performance profile collector pods to become ready: 1
2026-07-20T20:20:18.279655101Z Gathering data for ns/openshift-dns-operator...
2026-07-20T20:20:18.572131479Z Gathering data for ns/openshift-dns...
2026-07-20T20:20:18.610651748Z Done executing Istio gather script
2026-07-20T20:20:19.050802138Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T20:20:19.055119040Z Gathering data for ns/openshift-image-registry...
2026-07-20T20:20:19.101773906Z Waiting for performance profile collector pods to become ready: 2
2026-07-20T20:20:19.434607165Z Gathering data for ns/openshift-ingress-operator...
2026-07-20T20:20:19.721266123Z Gathering data for ns/openshift-ingress...
2026-07-20T20:20:19.990853515Z Gathering data for ns/openshift-ingress-canary...
2026-07-20T20:20:20.207370799Z Waiting for performance profile collector pods to become ready: 3
2026-07-20T20:20:20.310056322Z Gathering data for ns/openshift-insights...
2026-07-20T20:20:21.311648040Z Waiting for performance profile collector pods to become ready: 4
2026-07-20T20:20:21.970126636Z Gathering data for ns/openshift-jobset-operator...
2026-07-20T20:20:22.425167295Z Waiting for performance profile collector pods to become ready: 5
2026-07-20T20:20:22.723195656Z Gathering data for ns/openshift-config...
2026-07-20T20:20:22.948432658Z Gathering data for ns/openshift-config-managed...
2026-07-20T20:20:23.321419288Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-20T20:20:23.554532052Z Gathering data for ns/openshift-kube-apiserver...
2026-07-20T20:20:23.556952758Z Waiting for performance profile collector pods to become ready: 6
2026-07-20T20:20:23.762267406Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-20T20:20:23.996626975Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-20T20:20:24.057808304Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T20:20:24.243820587Z Gathering data for ns/openshift-kube-scheduler...
2026-07-20T20:20:24.459687770Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-20T20:20:24.686279458Z Waiting for performance profile collector pods to become ready: 7
2026-07-20T20:20:24.713373830Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-20T20:20:24.985074403Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-20T20:20:25.249497280Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-20T20:20:25.668611912Z Gathering data for ns/openshift-multus...
2026-07-20T20:20:25.786791992Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-20T20:20:26.006009767Z Collecting performance related data for node ip-10-0-129-241.ec2.internal
2026-07-20T20:20:26.013200817Z Collecting performance related data for node ip-10-0-142-28.ec2.internal
2026-07-20T20:20:26.020821256Z Collecting performance related data for node ip-10-0-139-133.ec2.internal
2026-07-20T20:20:26.986017742Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-20T20:20:28.060656927Z Gathering data for ns/openshift-host-network...
2026-07-20T20:20:28.371265237Z Gathering data for ns/openshift-network-diagnostics...
2026-07-20T20:20:28.869072824Z Gathering data for ns/openshift-network-node-identity...
2026-07-20T20:20:29.063777151Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T20:20:29.213820126Z Gathering data for ns/openshift-network-operator...
2026-07-20T20:20:29.538501750Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-20T20:20:29.762435648Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-20T20:20:30.218404011Z Gathering data for ns/openshift-apiserver-operator...
2026-07-20T20:20:30.479385612Z Gathering data for ns/openshift-apiserver...
2026-07-20T20:20:30.824885215Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-20T20:20:31.047619432Z Gathering data for ns/openshift-controller-manager...
2026-07-20T20:20:31.284162541Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-20T20:20:31.874136468Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-20T20:20:32.132960410Z Gathering data for ns/openshift-service-ca-operator...
2026-07-20T20:20:32.416774967Z Gathering data for ns/openshift-service-ca...
2026-07-20T20:20:32.732568595Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-20T20:20:33.552430239Z Wrote inspect data to must-gather.
2026-07-20T20:20:33.552476235Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T20:20:33.552476235Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-20T20:20:33.816215691Z Collecting kubelet logs for node ip-10-0-129-241.ec2.internal
2026-07-20T20:20:33.818071952Z Collecting kubelet logs for node ip-10-0-142-28.ec2.internal
2026-07-20T20:20:33.819909758Z Collecting kubelet logs for node ip-10-0-139-133.ec2.internal
2026-07-20T20:20:34.068571141Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T20:20:34.112638889Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4dd9x namespace
2026-07-20T20:20:34.117678414Z INFO: Node performance data collection complete.
2026-07-20T20:20:34.244513862Z error: the server doesn't have a resource type "clusters"
2026-07-20T20:20:34.412587607Z Caches written to disk