./mage -v ci:teste2e go: downloading k8s.io/klog/v2 v2.140.0 go: downloading k8s.io/utils v0.0.0-20260210185600-b8788abfbbc2 go: downloading k8s.io/kube-openapi v0.0.0-20260509192518-b540ad9def2b Running target: CI:TestE2E I0509 22:43:19.115247 26061 magefile.go:525] setting up new custom bundle for testing... I0509 22:43:19.427566 26061 util.go:512] found credentials for image ref quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1778366599-hnrk -> user: redhat-appstudio-qe+redhat_appstudio_quality Creating Tekton Bundle: - Added Pipeline: docker-build to image I0509 22:43:20.657441 26061 bundle.go:57] image digest for a new tekton bundle quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1778366599-hnrk: quay.io/redhat-appstudio-qe/test-images@sha256:94a72fbb69f0f78f20a1975c61256eb0bb5e4f34733a9b2bd0bf052bc3c6e7b1 I0509 22:43:20.657463 26061 magefile.go:531] To use the custom docker bundle locally, run below cmd: export CUSTOM_DOCKER_BUILD_PIPELINE_BUNDLE=quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1778366599-hnrk I0509 22:43:21.811941 26061 release_service_catalog.go:104] checking if repository is release-service-catalog I0509 22:43:21.811959 26061 integration_service.go:49] checking if repository is integration-service I0509 22:43:21.811963 26061 image_controller.go:49] checking if repository is image-controller I0509 22:43:21.811968 26061 build_service.go:50] checking if repository is build-service I0509 22:43:21.811971 26061 e2e_repo.go:347] checking if repository is e2e-tests I0509 22:43:21.811974 26061 e2e_repo.go:335] multi-platform tests and require sprayproxy registering are set to TRUE exec: git "diff" "--name-status" "upstream/main..HEAD" I0509 22:43:21.815106 26061 util.go:451] The following files, go.mod, go.sum, were changed! exec: go "install" "-mod=mod" "github.com/onsi/ginkgo/v2/ginkgo" go: downloading github.com/go-task/slim-sprig/v3 v3.0.0 go: downloading github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 I0509 22:43:25.084975 26061 install.go:226] cloning 'https://github.com/redhat-appstudio/infra-deployments' (ref: 'refs/heads/main', remote: 'upstream') Enumerating objects: 80723, done. Counting objects: 3% (1/31) Counting objects: 6% (2/31) Counting objects: 9% (3/31) Counting objects: 12% (4/31) Counting objects: 16% (5/31) Counting objects: 19% (6/31) Counting objects: 22% (7/31) Counting objects: 25% (8/31) Counting objects: 29% (9/31) Counting objects: 32% (10/31) Counting objects: 35% (11/31) Counting objects: 38% (12/31) Counting objects: 41% (13/31) Counting objects: 45% (14/31) Counting objects: 48% (15/31) Counting objects: 51% (16/31) Counting objects: 54% (17/31) Counting objects: 58% (18/31) Counting objects: 61% (19/31) Counting objects: 64% (20/31) Counting objects: 67% (21/31) Counting objects: 70% (22/31) Counting objects: 74% (23/31) Counting objects: 77% (24/31) Counting objects: 80% (25/31) Counting objects: 83% (26/31) Counting objects: 87% (27/31) Counting objects: 90% (28/31) Counting objects: 93% (29/31) Counting objects: 96% (30/31) Counting objects: 100% (31/31) Counting objects: 100% (31/31), done. Compressing objects: 3% (1/29) Compressing objects: 6% (2/29) Compressing objects: 10% (3/29) Compressing objects: 13% (4/29) Compressing objects: 17% (5/29) Compressing objects: 20% (6/29) Compressing objects: 24% (7/29) Compressing objects: 27% (8/29) Compressing objects: 31% (9/29) Compressing objects: 34% (10/29) Compressing objects: 37% (11/29) Compressing objects: 41% (12/29) Compressing objects: 44% (13/29) Compressing objects: 48% (14/29) Compressing objects: 51% (15/29) Compressing objects: 55% (16/29) Compressing objects: 58% (17/29) Compressing objects: 62% (18/29) Compressing objects: 65% (19/29) Compressing objects: 68% (20/29) Compressing objects: 72% (21/29) Compressing objects: 75% (22/29) Compressing objects: 79% (23/29) Compressing objects: 82% (24/29) Compressing objects: 86% (25/29) Compressing objects: 89% (26/29) Compressing objects: 93% (27/29) Compressing objects: 96% (28/29) Compressing objects: 100% (29/29) Compressing objects: 100% (29/29), done. Total 80723 (delta 11), reused 0 (delta 0), pack-reused 80692 (from 2) From https://github.com/redhat-appstudio/infra-deployments * branch main -> FETCH_HEAD Already up to date. ============================================================================= [2026-05-09 22:43:29] [STEP] Starting Konflux Cluster Bootstrap ============================================================================= [2026-05-09 22:43:29] [INFO] Mode: preview [2026-05-09 22:43:29] [INFO] Options: OBO=disabled, EAAS=disabled [2026-05-09 22:43:29] [INFO] Start time: 2026-05-09 22:43:29 UTC ============================================================================= [2026-05-09 22:43:29] [STEP] Phase 1: Deploying ArgoCD ============================================================================= ============================================================================= [2026-05-09 22:43:29] [STEP] Deploying OpenShift GitOps (ArgoCD) ============================================================================= [2026-05-09 22:43:29] [SUBSTEP] Verifying cluster-admin permissions [2026-05-09 22:43:29] [SUCCESS] User 'cluster-admin' has cluster-admin permissions [2026-05-09 22:43:29] [SUBSTEP] Installing OpenShift GitOps operator subscription clusterrole.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-application-controller created clusterrole.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-server created clusterrolebinding.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-application-controller created clusterrolebinding.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-server created subscription.operators.coreos.com/openshift-gitops-operator created [2026-05-09 22:43:30] [SUCCESS] GitOps operator subscription applied [2026-05-09 22:43:30] [SUBSTEP] Waiting for default ArgoCD project to be created [2026-05-09 22:43:30] [WAITING] Default project not yet available (5s/300s) [2026-05-09 22:43:35] [WAITING] Default project not yet available (10s/300s) [2026-05-09 22:43:40] [WAITING] Default project not yet available (15s/300s) [2026-05-09 22:43:46] [WAITING] Default project not yet available (20s/300s) [2026-05-09 22:43:51] [WAITING] Default project not yet available (25s/300s) [2026-05-09 22:43:56] [WAITING] Default project not yet available (30s/300s) [2026-05-09 22:44:01] [WAITING] Default project not yet available (35s/300s) [2026-05-09 22:44:06] [WAITING] Default project not yet available (40s/300s) [2026-05-09 22:44:11] [SUCCESS] Default ArgoCD project is available [2026-05-09 22:44:11] [SUBSTEP] Waiting for OpenShift GitOps route to be created [2026-05-09 22:44:11] [SUCCESS] OpenShift GitOps route is available [2026-05-09 22:44:11] [SUBSTEP] Configuring ArgoCD repo server resources and timeout argocd.argoproj.io/openshift-gitops patched [2026-05-09 22:44:12] [SUCCESS] Repo server configured: timeout=5m, cpu=100m, memory=100Mi [2026-05-09 22:44:12] [SUBSTEP] Configuring ArgoCD application controller resources argocd.argoproj.io/openshift-gitops patched [2026-05-09 22:44:12] [SUCCESS] Application controller configured: cpu=4, memory=4Gi [2026-05-09 22:44:12] [SUBSTEP] Switching ArgoCD route to re-encryption TLS argocd.argoproj.io/openshift-gitops patched [2026-05-09 22:44:12] [SUCCESS] Route TLS termination set to 'reencrypt' [2026-05-09 22:44:12] [SUBSTEP] Restarting ArgoCD server after TLS configuration change pod "openshift-gitops-server-6944966b75-vhpj5" deleted [2026-05-09 22:44:13] [SUCCESS] ArgoCD server pods deleted for restart [2026-05-09 22:44:13] [SUBSTEP] Granting admin role to authenticated users [2026-05-09 22:44:13] [INFO] Note: This should be updated once proper access policy is in place argocd.argoproj.io/openshift-gitops patched [2026-05-09 22:44:13] [SUCCESS] RBAC policy set: system:authenticated -> role:admin [2026-05-09 22:44:13] [SUBSTEP] Configuring PVC health check (WaitForFirstConsumer workaround) Warning: unknown field "spec.resourceCustomizations" argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:44:13] [SUCCESS] PVC health customization applied (Pending/Bound = Healthy) [2026-05-09 22:44:13] [SUBSTEP] Enabling Helm support in Kustomize builds argocd.argoproj.io/openshift-gitops patched [2026-05-09 22:44:13] [SUCCESS] Kustomize build options set: --enable-helm [2026-05-09 22:44:13] [SUBSTEP] Configuring ArgoCD to ignore aggregated roles in diff argocd.argoproj.io/openshift-gitops patched [2026-05-09 22:44:13] [SUCCESS] ignoreAggregatedRoles set to true [2026-05-09 22:44:13] [SUBSTEP] Setting ArgoCD tracking method to annotation argocd.argoproj.io/openshift-gitops patched [2026-05-09 22:44:13] [SUCCESS] Resource tracking method set to 'annotation' [2026-05-09 22:44:13] [SUBSTEP] Restarting GitOps server deployment deployment.apps/openshift-gitops-server restarted [2026-05-09 22:44:13] [SUCCESS] GitOps server restart initiated [2026-05-09 22:44:13] [INFO] Waiting for rollout to complete... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... deployment "openshift-gitops-server" successfully rolled out [2026-05-09 22:44:44] [SUCCESS] GitOps server rollout complete ============================================================================= [2026-05-09 22:44:44] [STEP] ArgoCD Access Information ============================================================================= [2026-05-09 22:44:44] [INFO] ArgoCD URL: https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:44:44] [INFO] Authentication: Use 'Login with OpenShift' button (OpenShift credentials) [2026-05-09 22:44:44] [SUBSTEP] Verifying ArgoCD route is accessible [2026-05-09 22:45:04] [WAITING] Waiting for route to respond (5s/120s) [2026-05-09 22:45:09] [SUCCESS] ArgoCD is accessible at https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [ARGOCD_DEPLOY_JSON] {"status":"success","url":"https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com","namespace":"openshift-gitops","auth_method":"openshift"} ============================================================================= [2026-05-09 22:45:09] [STEP] OpenShift GitOps Deployment Complete ============================================================================= ============================================================================= [2026-05-09 22:45:09] [STEP] Phase 2: Bootstrapping Host Cluster ============================================================================= [2026-05-09 22:45:09] [INFO] Bootstrapping host cluster components [2026-05-09 22:45:09] [INFO] No preview.env file found - using environment variables only [2026-05-09 22:45:09] [SUBSTEP] Configuring Quality Dashboard secrets Setting secrets for Quality Dashboard namespace/quality-dashboard created secret/quality-dashboard-secrets created [2026-05-09 22:45:13] [SUCCESS] Quality Dashboard secrets configured [2026-05-09 22:45:13] [SUBSTEP] Configuring CI Helper App secret Creating secret for CI Helper App namespace/ci-helper-app created secret/ci-helper-app-secrets created [2026-05-09 22:45:13] [SUCCESS] CI Helper App secret configured [2026-05-09 22:45:13] [SUCCESS] Host cluster bootstrap complete ============================================================================= [2026-05-09 22:45:13] [STEP] Phase 3: Bootstrapping Member Cluster ============================================================================= [2026-05-09 22:45:13] [INFO] Bootstrapping member cluster components [2026-05-09 22:45:13] [INFO] No preview.env file found - using environment variables only [2026-05-09 22:45:13] [SUBSTEP] Configuring Pipeline Service secrets Setting secrets for pipeline-service tekton-results namespace already exists, skipping creation tekton-logging namespace already exists, skipping creation namespace/product-kubearchive-logging created Creating DB secret secret/tekton-results-database created Creating S3 secret secret/tekton-results-s3 created Creating MinIO config secret/minio-storage-configuration created Creating S3 secret secret/tekton-results-s3 created Creating MinIO config MinIO config already exists, skipping creation Creating Postgres TLS certs secret/postgresql-tls created configmap/rds-root-crt created [2026-05-09 22:45:18] [SUCCESS] Pipeline Service secrets configured [2026-05-09 22:45:18] [SUBSTEP] Configuring GitHub secrets [2026-05-09 22:45:18] [INFO] GitHub token provided - creating secrets namespace/application-service created Creating a has secret from legacy token secret/has-github-token created [2026-05-09 22:45:19] [SUCCESS] GitHub secrets configured [2026-05-09 22:45:19] [SUBSTEP] Configuring Image Controller secrets [2026-05-09 22:45:19] [INFO] Quay organization: redhat-appstudio-qe Creating a secret with a token for Image Controller namespace/image-controller created secret/quaytoken created [2026-05-09 22:45:20] [SUCCESS] Image Controller secrets configured [2026-05-09 22:45:20] [SUCCESS] Member cluster bootstrap complete ============================================================================= [2026-05-09 22:45:20] [STEP] Phase 4: Bootstrapping Common Components ============================================================================= [2026-05-09 22:45:20] [INFO] Bootstrapping common cluster components [2026-05-09 22:45:20] [INFO] No preview.env file found at /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../hack/preview.env - using environment variables only [2026-05-09 22:45:20] [SUBSTEP] Configuring Docker Hub pull secret Configuring the cluster with a pull secret for Docker Hub Saved credentials for docker.io into /tmp/tmp.3ofNOQgmq4 secret/pull-secret data updated Saved credentials for docker.io into /tmp/tmp.3ofNOQgmq4 secret/docker-io-pull created [2026-05-09 22:45:21] [SUCCESS] Docker Hub pull secret configured [2026-05-09 22:45:21] [SUBSTEP] Configuring DORA metrics exporter secrets Setting secrets for Dora metrics exporter namespace/dora-metrics created secret/exporters-secret created [2026-05-09 22:45:21] [SUCCESS] DORA metrics exporter secrets configured [2026-05-09 22:45:21] [SUCCESS] Common cluster bootstrap complete ============================================================================= [2026-05-09 22:45:21] [STEP] Phase 5: Mode-Specific Deployment (preview) ============================================================================= [2026-05-09 22:45:21] [INFO] Deploying preview configuration ============================================================================= [2026-05-09 22:45:21] [STEP] Starting Konflux Preview Environment Setup ============================================================================= [2026-05-09 22:45:21] [INFO] Script: /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../hack/preview.sh [2026-05-09 22:45:21] [INFO] Options: OBO=false, GRAFANA=false, EAAS=false [2026-05-09 22:45:21] [INFO] Start time: 2026-05-09 22:45:21 UTC ============================================================================= [2026-05-09 22:45:21] [STEP] Cluster Context Information ============================================================================= [2026-05-09 22:45:22] [INFO] OpenShift Version: 4.18.9 [2026-05-09 22:45:22] [INFO] API Server: https://api.kx-4ccd113921.l5qz.p3.openshiftapps.com:443 [2026-05-09 22:45:22] [INFO] Cluster ID: 04b7f83c-45d4-432b-b66f-7f0cf55568d2 [2026-05-09 22:45:22] [INFO] Cluster Domain: apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:45:22] [INFO] Total Nodes: 3 [2026-05-09 22:45:22] [INFO] - Master nodes: 0 [2026-05-09 22:45:22] [INFO] - Worker nodes: 3 [2026-05-09 22:45:22] [SUCCESS] Cluster operators: All healthy [CLUSTER_CONTEXT_JSON] {"ocp_version":"4.18.9","api_server":"https://api.kx-4ccd113921.l5qz.p3.openshiftapps.com:443","cluster_id":"04b7f83c-45d4-432b-b66f-7f0cf55568d2","cluster_domain":"apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com","total_nodes":3,"master_nodes":0,"worker_nodes":3,"degraded_operators":0} Switched to a new branch 'preview-main-wexs' [2026-05-09 22:45:22] [SUCCESS] Git environment initialized [2026-05-09 22:45:22] [INFO] - Repository URL: https://github.com/redhat-appstudio-qe/infra-deployments.git [2026-05-09 22:45:22] [INFO] - Source branch: main [2026-05-09 22:45:22] [INFO] - Preview branch: preview-main-wexs [2026-05-09 22:45:22] [INFO] - GitHub org: redhat-appstudio-qe ============================================================================= [2026-05-09 22:45:22] [STEP] Patching ArgoCD application manifests to use fork repository ============================================================================= [2026-05-09 22:45:22] [INFO] Setting repoURL to: https://github.com/redhat-appstudio-qe/infra-deployments.git [2026-05-09 22:45:22] [INFO] Setting targetRevision to: preview-main-wexs [2026-05-09 22:45:22] [SUBSTEP] Patched: application-patch.yaml [2026-05-09 22:45:22] [SUBSTEP] Patched: application-set-patch.yaml [2026-05-09 22:45:22] [SUBSTEP] Patched: application-set-multisrc-src-1-patch.yaml [2026-05-09 22:45:22] [SUCCESS] All ArgoCD patch files updated ============================================================================= [2026-05-09 22:45:22] [STEP] Labeling cluster nodes for Konflux workloads ============================================================================= [2026-05-09 22:45:22] [INFO] Reference: https://github.com/redhat-appstudio/infra-deployments/pull/4415 [2026-05-09 22:45:23] [INFO] Found 3 nodes to label with 'konflux-ci.dev/workload=konflux-tenants' [2026-05-09 22:45:23] [SUBSTEP] Labeling node/ip-10-0-137-137.ec2.internal node/ip-10-0-137-137.ec2.internal labeled [2026-05-09 22:45:23] [SUCCESS] Successfully labeled node/ip-10-0-137-137.ec2.internal [2026-05-09 22:45:23] [SUBSTEP] Labeling node/ip-10-0-157-249.ec2.internal node/ip-10-0-157-249.ec2.internal labeled [2026-05-09 22:45:23] [SUCCESS] Successfully labeled node/ip-10-0-157-249.ec2.internal [2026-05-09 22:45:23] [SUBSTEP] Labeling node/ip-10-0-169-212.ec2.internal node/ip-10-0-169-212.ec2.internal labeled [2026-05-09 22:45:23] [SUCCESS] Successfully labeled node/ip-10-0-169-212.ec2.internal [2026-05-09 22:45:23] [SUBSTEP] Verifying labels on all nodes... [2026-05-09 22:45:23] [SUCCESS] All 3 nodes labeled and verified successfully ============================================================================= [2026-05-09 22:45:23] [STEP] Checking OCP version for Kueue compatibility ============================================================================= [2026-05-09 22:45:23] [INFO] Detected OpenShift Container Platform version: 4.18.9 (minor: 18) [2026-05-09 22:45:23] [SUCCESS] OCP version 4.18.9 meets Kueue requirements - Kueue will be deployed ============================================================================= [2026-05-09 22:45:23] [STEP] Configuring GitHub organization ============================================================================= [2026-05-09 22:45:23] [INFO] Setting GitHub org to: redhat-appstudio-qe [2026-05-09 22:45:23] [INFO] Configuring GitHub organization for Application Service (HAS) [2026-05-09 22:45:23] [INFO] - Target org: redhat-appstudio-qe [2026-05-09 22:45:23] [INFO] - Config file: /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../components/has/base/kustomization.yaml [2026-05-09 22:45:23] [SUCCESS] GitHub organization set to 'redhat-appstudio-qe' [2026-05-09 22:45:23] [SUCCESS] Verified: Configuration updated correctly [2026-05-09 22:45:23] [SUCCESS] GitHub organization configured ============================================================================= [2026-05-09 22:45:23] [STEP] Configuring Rekor server hostname ============================================================================= [2026-05-09 22:45:23] [INFO] Cluster domain: apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:45:23] [INFO] Rekor server hostname: rekor.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:45:23] [SUCCESS] Rekor server hostname configured ============================================================================= [2026-05-09 22:45:23] [STEP] Applying service image overrides from environment variables ============================================================================= [2026-05-09 22:45:23] [INFO] No service image overrides configured - using default images ============================================================================= [2026-05-09 22:45:23] [STEP] Committing and pushing preview changes ============================================================================= [preview-main-wexs f6c2288b9] Preview mode, do not merge into main 8 files changed, 15 insertions(+), 19 deletions(-) remote: remote: Create a pull request for 'preview-main-wexs' on GitHub by visiting: remote: https://github.com/redhat-appstudio-qe/infra-deployments/pull/new/preview-main-wexs remote: To https://github.com/redhat-appstudio-qe/infra-deployments.git * [new branch] preview-main-wexs -> preview-main-wexs branch 'preview-main-wexs' set up to track 'qe/preview-main-wexs'. [2026-05-09 22:45:24] [SUCCESS] Preview changes committed and pushed to qe/preview-main-wexs ============================================================================= [2026-05-09 22:45:24] [STEP] Deploying ArgoCD applications ============================================================================= [2026-05-09 22:45:24] [SUBSTEP] Applying root Application from: /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../argo-cd-apps/app-of-app-sets/development application.argoproj.io/all-application-sets created [2026-05-09 22:45:25] [SUCCESS] Root Application 'all-application-sets' created [2026-05-09 22:45:25] [SUBSTEP] Waiting for 'all-application-sets' to become Healthy and Synced [2026-05-09 22:45:25] [WAITING] Root application status: ' ' (target: 'Healthy Synced') - 5s elapsed [2026-05-09 22:45:30] [WAITING] Root application status: ' ' (target: 'Healthy Synced') - 10s elapsed [2026-05-09 22:45:35] [WAITING] Root application status: 'Healthy OutOfSync' (target: 'Healthy Synced') - 15s elapsed [2026-05-09 22:45:41] [SUCCESS] Root application 'all-application-sets' is Healthy and Synced [2026-05-09 22:45:41] [SUBSTEP] Triggering hard refresh on all ArgoCD applications [2026-05-09 22:45:41] [INFO] Found 43 applications to refresh application.argoproj.io/trust-manager-in-cluster-local patched application.argoproj.io/disable-csvcopy-in-cluster-local patched application.argoproj.io/konflux-info-in-cluster-local patched application.argoproj.io/knative-eventing-in-cluster-local patched application.argoproj.io/disaster-recovery-in-cluster-local patched application.argoproj.io/dora-metrics-in-cluster-local patched application.argoproj.io/perf-team-prometheus-reader-in-cluster-local patched application.argoproj.io/repository-validator-in-cluster-local patched application.argoproj.io/monitoring-workload-prometheus-in-cluster-local patched application.argoproj.io/image-controller-in-cluster-local patched application.argoproj.io/policies-in-cluster-local patched application.argoproj.io/monitoring-registry-in-cluster-local patched application.argoproj.io/kubearchive-in-cluster-local patched application.argoproj.io/konflux-rbac-in-cluster-local patched application.argoproj.io/project-controller-in-cluster-local patched application.argoproj.io/build-service-in-cluster-local patched application.argoproj.io/release-in-cluster-local patched application.argoproj.io/build-templates-in-cluster-local patched application.argoproj.io/has-in-cluster-local patched application.argoproj.io/pipeline-service-in-cluster-local patched application.argoproj.io/squid-in-cluster-local patched application.argoproj.io/integration-in-cluster-local patched application.argoproj.io/all-application-sets patched application.argoproj.io/tempo-in-cluster-local patched application.argoproj.io/smee-client-in-cluster-local patched application.argoproj.io/tracing-workload-otel-collector-in-cluster-local patched application.argoproj.io/enterprise-contract-in-cluster-local patched application.argoproj.io/konflux-kite-in-cluster-local patched application.argoproj.io/image-rbac-proxy-in-cluster-local patched application.argoproj.io/vector-kubearchive-log-collector-in-cluster-local patched (no change) application.argoproj.io/tracing-workload-tracing-in-cluster-local patched application.argoproj.io/kyverno-in-cluster-local patched application.argoproj.io/cert-manager-in-cluster-local patched application.argoproj.io/internal-services-in-cluster-local patched application.argoproj.io/kueue-in-cluster-local patched application.argoproj.io/multi-platform-controller-in-cluster-local patched application.argoproj.io/crossplane-control-plane-in-cluster-local patched application.argoproj.io/monitoring-workload-custom-kube-state-metrics-in-cluster-local patched application.argoproj.io/vector-tekton-logs-collector-in-cluster-local patched application.argoproj.io/mintmaker-in-cluster-local patched application.argoproj.io/monitoring-cardinality-in-cluster-local patched application.argoproj.io/application-api-in-cluster-local patched application.argoproj.io/backup-in-cluster-local patched [2026-05-09 22:45:48] [SUCCESS] Hard refresh triggered on all 43 applications [2026-05-09 22:45:48] [SUBSTEP] Waiting for refresh operations to complete [2026-05-09 22:45:48] [PROGRESS] Refresh: 0/43 complete | 43 still refreshing (5s elapsed) [2026-05-09 22:45:53] [PROGRESS] Refresh: 0/43 complete | 43 still refreshing (10s elapsed) [2026-05-09 22:45:58] [PROGRESS] Refresh: 4/43 complete | 39 still refreshing (15s elapsed) [2026-05-09 22:46:04] [PROGRESS] Refresh: 5/43 complete | 38 still refreshing (20s elapsed) [2026-05-09 22:46:09] [PROGRESS] Refresh: 14/43 complete | 29 still refreshing (25s elapsed) [2026-05-09 22:46:14] [PROGRESS] Refresh: 29/43 complete | 14 still refreshing (30s elapsed) [2026-05-09 22:46:19] [PROGRESS] Refresh: 31/43 complete | 12 still refreshing (35s elapsed) [2026-05-09 22:46:25] [PROGRESS] Refresh: 37/43 complete | 6 still refreshing (40s elapsed) [2026-05-09 22:46:30] [PROGRESS] Refresh: 39/43 complete | 4 still refreshing (45s elapsed) [2026-05-09 22:46:35] [PROGRESS] Refresh: 39/43 complete | 4 still refreshing (50s elapsed) [2026-05-09 22:46:41] [PROGRESS] Refresh: 40/43 complete | 3 still refreshing (55s elapsed) [2026-05-09 22:46:46] [PROGRESS] Refresh: 40/43 complete | 3 still refreshing (60s elapsed) [2026-05-09 22:46:51] [PROGRESS] Refresh: 41/43 complete | 2 still refreshing (65s elapsed) [2026-05-09 22:46:56] [PROGRESS] Refresh: 41/43 complete | 2 still refreshing (70s elapsed) [2026-05-09 22:47:02] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (75s elapsed) [2026-05-09 22:47:07] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (80s elapsed) [2026-05-09 22:47:12] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (85s elapsed) [2026-05-09 22:47:18] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (90s elapsed) [2026-05-09 22:47:23] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (95s elapsed) [2026-05-09 22:47:28] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (100s elapsed) [2026-05-09 22:47:33] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (105s elapsed) [2026-05-09 22:47:39] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (110s elapsed) [2026-05-09 22:47:44] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (115s elapsed) [2026-05-09 22:47:49] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (120s elapsed) [2026-05-09 22:47:55] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (125s elapsed) [2026-05-09 22:48:00] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (130s elapsed) [2026-05-09 22:48:05] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (135s elapsed) [2026-05-09 22:48:11] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (140s elapsed) [2026-05-09 22:48:16] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (145s elapsed) [2026-05-09 22:48:21] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (150s elapsed) [2026-05-09 22:48:27] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (155s elapsed) [2026-05-09 22:48:32] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (160s elapsed) [2026-05-09 22:48:37] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (165s elapsed) [2026-05-09 22:48:43] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (170s elapsed) [2026-05-09 22:48:48] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (175s elapsed) [2026-05-09 22:48:53] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (180s elapsed) [2026-05-09 22:48:59] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (185s elapsed) [2026-05-09 22:49:04] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (190s elapsed) [2026-05-09 22:49:10] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (195s elapsed) [2026-05-09 22:49:15] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (200s elapsed) [2026-05-09 22:49:20] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (205s elapsed) [2026-05-09 22:49:26] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (210s elapsed) [2026-05-09 22:49:31] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (215s elapsed) [2026-05-09 22:49:36] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (220s elapsed) [2026-05-09 22:49:41] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (225s elapsed) [2026-05-09 22:49:47] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (230s elapsed) [2026-05-09 22:49:52] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (235s elapsed) [2026-05-09 22:49:57] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (240s elapsed) [2026-05-09 22:50:03] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (245s elapsed) [2026-05-09 22:50:08] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (250s elapsed) [2026-05-09 22:50:13] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (255s elapsed) [2026-05-09 22:50:19] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (260s elapsed) [2026-05-09 22:50:24] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (265s elapsed) [2026-05-09 22:50:29] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (270s elapsed) [2026-05-09 22:50:35] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (275s elapsed) [2026-05-09 22:50:40] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (280s elapsed) [2026-05-09 22:50:45] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (285s elapsed) [2026-05-09 22:50:51] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (290s elapsed) [2026-05-09 22:50:56] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (295s elapsed) [2026-05-09 22:51:01] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (300s elapsed) [2026-05-09 22:51:07] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (305s elapsed) [2026-05-09 22:51:12] [SUCCESS] All 43 applications refreshed ============================================================================= [2026-05-09 22:51:12] [STEP] Waiting for all ArgoCD applications to sync and become healthy ============================================================================= [2026-05-09 22:51:12] [INFO] Timeout: 2700 seconds (45 minutes) [2026-05-09 22:51:12] [PROGRESS] Applications: 20/44 ready | 24 pending (0m 0s elapsed) [2026-05-09 22:51:12] [INFO] Pending: application-api-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,enterprise-contract-in-cluster-local,integration-in-cluster-local,internal-services-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-cardinality-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-custom-kube-state-metrics-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,squid-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-05-09 22:51:12] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 22:51:22] [PROGRESS] Applications: 20/44 ready | 24 pending (0m 10s elapsed) [2026-05-09 22:51:22] [INFO] Pending: application-api-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,enterprise-contract-in-cluster-local,integration-in-cluster-local,internal-services-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-cardinality-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-custom-kube-state-metrics-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,squid-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-05-09 22:51:22] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 22:51:33] [PROGRESS] Applications: 25/44 ready | 19 pending (0m 20s elapsed) [2026-05-09 22:51:33] [INFO] Pending: application-api-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,enterprise-contract-in-cluster-local,integration-in-cluster-local,internal-services-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-cardinality-in-cluster-local,monitoring-registry-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,squid-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-05-09 22:51:33] [WARN] Found applications in Unknown state (not Progressing), investigating... [2026-05-09 22:51:33] [ERROR] Application 'vector-kubearchive-log-collector-in-cluster-local' is in Unknown state without 'context deadline exceeded' [2026-05-09 22:51:33] [INFO] ├─ App: vector-kubearchive-log-collector-in-cluster-local [2026-05-09 22:51:33] [INFO] │ ├─ Sync Status: Unknown [2026-05-09 22:51:33] [INFO] │ ├─ Health Status: Healthy [2026-05-09 22:51:33] [INFO] │ ├─ Out-of-sync resources: 17 [2026-05-09 22:51:33] [WARN] │ └─ Conditions: [2026-05-09 22:51:33] [WARN] │ └─ [ComparisonError] Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = `kustomize build /components/vector-kubearchive-log-collector/development --enable-helm --helm-kube-version 1.31 --helm-api-versions admissionregistration.k8s.io/v1 --helm-api-versions admissionregistration.k8s.io/v1/MutatingWebhookConfiguration --helm-api-versions admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy --helm-api-versions admissionregistration.k8s.io/v1/ValidatingAdmissionPolicyBinding --helm-api-versions admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration --helm-api-versions admissionregistration.k8s.io/v1beta1 --helm-api-versions admissionregistration.k8s.io/v1beta1/ValidatingAdmissionPolicy --helm-api-versions admissionregistration.k8s.io/v1beta1/ValidatingAdmissionPolicyBinding --helm-api-versions apiextensions.crossplane.io/v1 --helm-api-versions apiextensions.crossplane.io/v1/CompositeResourceDefinition --helm-api-versions apiextensions.crossplane.io/v1/Composition --helm-api-versions apiextensions.crossplane.io/v1/CompositionRevision --helm-api-versions apiextensions.crossplane.io/v1alpha1 --helm-api-versions apiextensions.crossplane.io/v1alpha1/ManagedResourceActivationPolicy --helm-api-versions apiextensions.crossplane.io/v1alpha1/ManagedResourceDefinition --helm-api-versions apiextensions.crossplane.io/v1beta1 --helm-api-versions apiextensions.crossplane.io/v1beta1/EnvironmentConfig --helm-api-versions apiextensions.crossplane.io/v2 --helm-api-versions apiextensions.crossplane.io/v2/CompositeResourceDefinition --helm-api-versions apiextensions.k8s.io/v1 --helm-api-versions apiextensions.k8s.io/v1/CustomResourceDefinition --helm-api-versions apiregistration.k8s.io/v1 --helm-api-versions apiregistration.k8s.io/v1/APIService --helm-api-versions apiserver.openshift.io/v1 --helm-api-versions apiserver.openshift.io/v1/APIRequestCount --helm-api-versions apps.openshift.io/v1 --helm-api-versions apps.openshift.io/v1/DeploymentConfig --helm-api-versions apps/v1 --helm-api-versions apps/v1/ControllerRevision --helm-api-versions apps/v1/DaemonSet --helm-api-versions apps/v1/Deployment --helm-api-versions apps/v1/ReplicaSet --helm-api-versions apps/v1/StatefulSet --helm-api-versions appstudio.redhat.com/v1alpha1 --helm-api-versions appstudio.redhat.com/v1alpha1/ImageRepository --helm-api-versions appstudio.redhat.com/v1alpha1/Release --helm-api-versions appstudio.redhat.com/v1alpha1/ReleasePlan --helm-api-versions appstudio.redhat.com/v1alpha1/ReleasePlanAdmission --helm-api-versions appstudio.redhat.com/v1alpha1/ReleaseServiceConfig --helm-api-versions argoproj.io/v1alpha1 --helm-api-versions argoproj.io/v1alpha1/AnalysisRun --helm-api-versions argoproj.io/v1alpha1/AnalysisTemplate --helm-api-versions argoproj.io/v1alpha1/AppProject --helm-api-versions argoproj.io/v1alpha1/Application --helm-api-versions argoproj.io/v1alpha1/ApplicationSet --helm-api-versions argoproj.io/v1alpha1/ArgoCD --helm-api-versions argoproj.io/v1alpha1/ClusterAnalysisTemplate --helm-api-versions argoproj.io/v1alpha1/Experiment --helm-api-versions argoproj.io/v1alpha1/NotificationsConfiguration --helm-api-versions argoproj.io/v1alpha1/Rollout --helm-api-versions argoproj.io/v1alpha1/RolloutManager --helm-api-versions argoproj.io/v1beta1 --helm-api-versions argoproj.io/v1beta1/ArgoCD --helm-api-versions authorization.openshift.io/v1 --helm-api-versions authorization.openshift.io/v1/RoleBindingRestriction --helm-api-versions autoscaling/v1 --helm-api-versions autoscaling/v1/HorizontalPodAutoscaler --helm-api-versions autoscaling/v2 --helm-api-versions autoscaling/v2/HorizontalPodAutoscaler --helm-api-versions batch/v1 --helm-api-versions batch/v1/CronJob --helm-api-versions batch/v1/Job --helm-api-versions build.openshift.io/v1 --helm-api-versions build.openshift.io/v1/Build --helm-api-versions build.openshift.io/v1/BuildConfig --helm-api-versions certificates.k8s.io/v1 --helm-api-versions certificates.k8s.io/v1/CertificateSigningRequest --helm-api-versions cloud.network.openshift.io/v1 --helm-api-versions cloud.network.openshift.io/v1/CloudPrivateIPConfig --helm-api-versions cloudcredential.openshift.io/v1 --helm-api-versions cloudcredential.openshift.io/v1/CredentialsRequest --helm-api-versions cluster.open-cluster-management.io/v1alpha1 --helm-api-versions cluster.open-cluster-management.io/v1alpha1/ClusterClaim --helm-api-versions config.openshift.io/v1 --helm-api-versions config.openshift.io/v1/APIServer --helm-api-versions config.openshift.io/v1/Authentication --helm-api-versions config.openshift.io/v1/Build --helm-api-versions config.openshift.io/v1/ClusterOperator --helm-api-versions config.openshift.io/v1/ClusterVersion --helm-api-versions config.openshift.io/v1/Console --helm-api-versions config.openshift.io/v1/DNS --helm-api-versions config.openshift.io/v1/FeatureGate --helm-api-versions config.openshift.io/v1/Image --helm-api-versions config.openshift.io/v1/ImageContentPolicy --helm-api-versions config.openshift.io/v1/ImageDigestMirrorSet --helm-api-versions config.openshift.io/v1/ImageTagMirrorSet --helm-api-versions config.openshift.io/v1/Infrastructure --helm-api-versions config.openshift.io/v1/Ingress --helm-api-versions config.openshift.io/v1/Network --helm-api-versions config.openshift.io/v1/Node --helm-api-versions config.openshift.io/v1/OAuth --helm-api-versions config.openshift.io/v1/OperatorHub --helm-api-versions config.openshift.io/v1/Project --helm-api-versions config.openshift.io/v1/Proxy --helm-api-versions config.openshift.io/v1/Scheduler --helm-api-versions console.openshift.io/v1 --helm-api-versions console.openshift.io/v1/ConsoleCLIDownload --helm-api-versions console.openshift.io/v1/ConsoleExternalLogLink --helm-api-versions console.openshift.io/v1/ConsoleLink --helm-api-versions console.openshift.io/v1/ConsoleNotification --helm-api-versions console.openshift.io/v1/ConsolePlugin --helm-api-versions console.openshift.io/v1/ConsoleQuickStart --helm-api-versions console.openshift.io/v1/ConsoleSample --helm-api-versions console.openshift.io/v1/ConsoleYAMLSample --helm-api-versions console.openshift.io/v1alpha1 --helm-api-versions console.openshift.io/v1alpha1/ConsolePlugin --helm-api-versions controlplane.operator.openshift.io/v1alpha1 --helm-api-versions controlplane.operator.openshift.io/v1alpha1/PodNetworkConnectivityCheck --helm-api-versions coordination.k8s.io/v1 --helm-api-versions coordination.k8s.io/v1/Lease --helm-api-versions discovery.k8s.io/v1 --helm-api-versions discovery.k8s.io/v1/EndpointSlice --helm-api-versions eventing.knative.dev/v1 --helm-api-versions eventing.knative.dev/v1/Broker --helm-api-versions eventing.knative.dev/v1/Trigger --helm-api-versions eventing.knative.dev/v1alpha1 --helm-api-versions eventing.knative.dev/v1alpha1/EventPolicy --helm-api-versions eventing.knative.dev/v1beta1 --helm-api-versions eventing.knative.dev/v1beta1/EventType --helm-api-versions eventing.knative.dev/v1beta2 --helm-api-versions eventing.knative.dev/v1beta2/EventType --helm-api-versions eventing.knative.dev/v1beta3 --helm-api-versions eventing.knative.dev/v1beta3/EventType --helm-api-versions events.k8s.io/v1 --helm-api-versions events.k8s.io/v1/Event --helm-api-versions flowcontrol.apiserver.k8s.io/v1 --helm-api-versions flowcontrol.apiserver.k8s.io/v1/FlowSchema --helm-api-versions flowcontrol.apiserver.k8s.io/v1/PriorityLevelConfiguration --helm-api-versions flowcontrol.apiserver.k8s.io/v1beta3 --helm-api-versions flowcontrol.apiserver.k8s.io/v1beta3/FlowSchema --helm-api-versions flowcontrol.apiserver.k8s.io/v1beta3/PriorityLevelConfiguration --helm-api-versions flows.knative.dev/v1 --helm-api-versions flows.knative.dev/v1/Parallel --helm-api-versions flows.knative.dev/v1/Sequence --helm-api-versions helm.openshift.io/v1beta1 --helm-api-versions helm.openshift.io/v1beta1/HelmChartRepository --helm-api-versions helm.openshift.io/v1beta1/ProjectHelmChartRepository --helm-api-versions image.openshift.io/v1 --helm-api-versions image.openshift.io/v1/Image --helm-api-versions image.openshift.io/v1/ImageStream --helm-api-versions imageregistry.operator.openshift.io/v1 --helm-api-versions imageregistry.operator.openshift.io/v1/Config --helm-api-versions imageregistry.operator.openshift.io/v1/ImagePruner --helm-api-versions ingress.operator.openshift.io/v1 --helm-api-versions ingress.operator.openshift.io/v1/DNSRecord --helm-api-versions k8s.cni.cncf.io/v1 --helm-api-versions k8s.cni.cncf.io/v1/NetworkAttachmentDefinition --helm-api-versions k8s.cni.cncf.io/v1alpha1 --helm-api-versions k8s.cni.cncf.io/v1alpha1/IPAMClaim --helm-api-versions k8s.ovn.org/v1 --helm-api-versions k8s.ovn.org/v1/AdminPolicyBasedExternalRoute --helm-api-versions k8s.ovn.org/v1/ClusterUserDefinedNetwork --helm-api-versions k8s.ovn.org/v1/EgressFirewall --helm-api-versions k8s.ovn.org/v1/EgressIP --helm-api-versions k8s.ovn.org/v1/EgressQoS --helm-api-versions k8s.ovn.org/v1/EgressService --helm-api-versions k8s.ovn.org/v1/UserDefinedNetwork --helm-api-versions machineconfiguration.openshift.io/v1 --helm-api-versions machineconfiguration.openshift.io/v1/ControllerConfig --helm-api-versions messaging.knative.dev/v1 --helm-api-versions messaging.knative.dev/v1/Channel --helm-api-versions messaging.knative.dev/v1/InMemoryChannel --helm-api-versions messaging.knative.dev/v1/Subscription --helm-api-versions migration.k8s.io/v1alpha1 --helm-api-versions migration.k8s.io/v1alpha1/StorageState --helm-api-versions migration.k8s.io/v1alpha1/StorageVersionMigration --helm-api-versions monitoring.coreos.com/v1 --helm-api-versions monitoring.coreos.com/v1/Alertmanager --helm-api-versions monitoring.coreos.com/v1/PodMonitor --helm-api-versions monitoring.coreos.com/v1/Probe --helm-api-versions monitoring.coreos.com/v1/Prometheus --helm-api-versions monitoring.coreos.com/v1/PrometheusRule --helm-api-versions monitoring.coreos.com/v1/ServiceMonitor --helm-api-versions monitoring.coreos.com/v1/ThanosRuler --helm-api-versions monitoring.coreos.com/v1alpha1 --helm-api-versions monitoring.coreos.com/v1alpha1/AlertmanagerConfig --helm-api-versions monitoring.coreos.com/v1beta1 --helm-api-versions monitoring.coreos.com/v1beta1/AlertmanagerConfig --helm-api-versions monitoring.openshift.io/v1 --helm-api-versions monitoring.openshift.io/v1/AlertRelabelConfig --helm-api-versions monitoring.openshift.io/v1/AlertingRule --helm-api-versions network.operator.openshift.io/v1 --helm-api-versions network.operator.openshift.io/v1/EgressRouter --helm-api-versions network.operator.openshift.io/v1/OperatorPKI --helm-api-versions networking.k8s.io/v1 --helm-api-versions networking.k8s.io/v1/Ingress --helm-api-versions networking.k8s.io/v1/IngressClass --helm-api-versions networking.k8s.io/v1/NetworkPolicy --helm-api-versions node.k8s.io/v1 --helm-api-versions node.k8s.io/v1/RuntimeClass --helm-api-versions oadp.openshift.io/v1alpha1 --helm-api-versions oadp.openshift.io/v1alpha1/CloudStorage --helm-api-versions oadp.openshift.io/v1alpha1/DataProtectionApplication --helm-api-versions oauth.openshift.io/v1 --helm-api-versions oauth.openshift.io/v1/OAuthAccessToken --helm-api-versions oauth.openshift.io/v1/OAuthAuthorizeToken --helm-api-versions oauth.openshift.io/v1/OAuthClient --helm-api-versions oauth.openshift.io/v1/OAuthClientAuthorization --helm-api-versions oauth.openshift.io/v1/UserOAuthAccessToken --helm-api-versions operator.openshift.io/v1 --helm-api-versions operator.openshift.io/v1/CSISnapshotController --helm-api-versions operator.openshift.io/v1/CloudCredential --helm-api-versions operator.openshift.io/v1/ClusterCSIDriver --helm-api-versions operator.openshift.io/v1/Config --helm-api-versions operator.openshift.io/v1/Console --helm-api-versions operator.openshift.io/v1/DNS --helm-api-versions operator.openshift.io/v1/Etcd --helm-api-versions operator.openshift.io/v1/IngressController --helm-api-versions operator.openshift.io/v1/InsightsOperator --helm-api-versions operator.openshift.io/v1/KubeAPIServer --helm-api-versions operator.openshift.io/v1/KubeControllerManager --helm-api-versions operator.openshift.io/v1/KubeScheduler --helm-api-versions operator.openshift.io/v1/KubeStorageVersionMigrator --helm-api-versions operator.openshift.io/v1/MachineConfiguration --helm-api-versions operator.openshift.io/v1/Network --helm-api-versions operator.openshift.io/v1/OpenShiftAPIServer --helm-api-versions operator.openshift.io/v1/OpenShiftControllerManager --helm-api-versions operator.openshift.io/v1/ServiceCA --helm-api-versions operator.openshift.io/v1/Storage --helm-api-versions operator.openshift.io/v1alpha1 --helm-api-versions operator.openshift.io/v1alpha1/ImageContentSourcePolicy --helm-api-versions operators.coreos.com/v1 --helm-api-versions operators.coreos.com/v1/OLMConfig --helm-api-versions operators.coreos.com/v1/Operator --helm-api-versions operators.coreos.com/v1/OperatorCondition --helm-api-versions operators.coreos.com/v1/OperatorGroup --helm-api-versions operators.coreos.com/v1alpha1 --helm-api-versions operators.coreos.com/v1alpha1/CatalogSource --helm-api-versions operators.coreos.com/v1alpha1/ClusterServiceVersion --helm-api-versions operators.coreos.com/v1alpha1/InstallPlan --helm-api-versions operators.coreos.com/v1alpha1/Subscription --helm-api-versions operators.coreos.com/v1alpha2 --helm-api-versions operators.coreos.com/v1alpha2/OperatorGroup --helm-api-versions operators.coreos.com/v2 --helm-api-versions operators.coreos.com/v2/OperatorCondition --helm-api-versions package-operator.run/v1alpha1 --helm-api-versions package-operator.run/v1alpha1/ClusterObjectDeployment --helm-api-versions package-operator.run/v1alpha1/ClusterObjectSet --helm-api-versions package-operator.run/v1alpha1/ClusterObjectSetPhase --helm-api-versions package-operator.run/v1alpha1/ClusterObjectSlice --helm-api-versions package-operator.run/v1alpha1/ClusterObjectTemplate --helm-api-versions package-operator.run/v1alpha1/ClusterPackage --helm-api-versions package-operator.run/v1alpha1/HostedClusterPackage --helm-api-versions package-operator.run/v1alpha1/ObjectDeployment --helm-api-versions package-operator.run/v1alpha1/ObjectSet --helm-api-versions package-operator.run/v1alpha1/ObjectSetPhase --helm-api-versions package-operator.run/v1alpha1/ObjectSlice --helm-api-versions package-operator.run/v1alpha1/ObjectTemplate --helm-api-versions package-operator.run/v1alpha1/Package --helm-api-versions pipelines.openshift.io/v1alpha1 --helm-api-versions pipelines.openshift.io/v1alpha1/GitopsService --helm-api-versions policy.networking.k8s.io/v1alpha1 --helm-api-versions policy.networking.k8s.io/v1alpha1/AdminNetworkPolicy --helm-api-versions policy.networking.k8s.io/v1alpha1/BaselineAdminNetworkPolicy --helm-api-versions policy/v1 --helm-api-versions policy/v1/PodDisruptionBudget --helm-api-versions projctl.konflux.dev/v1beta1 --helm-api-versions projctl.konflux.dev/v1beta1/Project --helm-api-versions projctl.konflux.dev/v1beta1/ProjectDevelopmentStream --helm-api-versions projctl.konflux.dev/v1beta1/ProjectDevelopmentStreamTemplate --helm-api-versions project.openshift.io/v1 --helm-api-versions project.openshift.io/v1/Project --helm-api-versions quota.openshift.io/v1 --helm-api-versions quota.openshift.io/v1/ClusterResourceQuota --helm-api-versions rbac.authorization.k8s.io/v1 --helm-api-versions rbac.authorization.k8s.io/v1/ClusterRole --helm-api-versions rbac.authorization.k8s.io/v1/ClusterRoleBinding --helm-api-versions rbac.authorization.k8s.io/v1/Role --helm-api-versions rbac.authorization.k8s.io/v1/RoleBinding --helm-api-versions route.openshift.io/v1 --helm-api-versions route.openshift.io/v1/Route --helm-api-versions samples.operator.openshift.io/v1 --helm-api-versions samples.operator.openshift.io/v1/Config --helm-api-versions scheduling.k8s.io/v1 --helm-api-versions scheduling.k8s.io/v1/PriorityClass --helm-api-versions security.internal.openshift.io/v1 --helm-api-versions security.internal.openshift.io/v1/RangeAllocation --helm-api-versions security.openshift.io/v1 --helm-api-versions security.openshift.io/v1/RangeAllocation --helm-api-versions security.openshift.io/v1/SecurityContextConstraints --helm-api-versions sinks.knative.dev/v1alpha1 --helm-api-versions sinks.knative.dev/v1alpha1/JobSink --helm-api-versions snapshot.storage.k8s.io/v1 --helm-api-versions snapshot.storage.k8s.io/v1/VolumeSnapshot --helm-api-versions snapshot.storage.k8s.io/v1/VolumeSnapshotClass --helm-api-versions snapshot.storage.k8s.io/v1/VolumeSnapshotContent --helm-api-versions sources.knative.dev/v1 --helm-api-versions sources.knative.dev/v1/ApiServerSource --helm-api-versions sources.knative.dev/v1/ContainerSource --helm-api-versions sources.knative.dev/v1/PingSource --helm-api-versions sources.knative.dev/v1/SinkBinding --helm-api-versions sources.knative.dev/v1beta2 --helm-api-versions sources.knative.dev/v1beta2/PingSource --helm-api-versions storage.k8s.io/v1 --helm-api-versions storage.k8s.io/v1/CSIDriver --helm-api-versions storage.k8s.io/v1/CSINode --helm-api-versions storage.k8s.io/v1/CSIStorageCapacity --helm-api-versions storage.k8s.io/v1/StorageClass --helm-api-versions storage.k8s.io/v1/VolumeAttachment --helm-api-versions template.openshift.io/v1 --helm-api-versions template.openshift.io/v1/BrokerTemplateInstance --helm-api-versions template.openshift.io/v1/Template --helm-api-versions template.openshift.io/v1/TemplateInstance --helm-api-versions tuned.openshift.io/v1 --helm-api-versions tuned.openshift.io/v1/Profile --helm-api-versions tuned.openshift.io/v1/Tuned --helm-api-versions user.openshift.io/v1 --helm-api-versions user.openshift.io/v1/Group --helm-api-versions user.openshift.io/v1/Identity --helm-api-versions user.openshift.io/v1/User --helm-api-versions v1 --helm-api-versions v1/ConfigMap --helm-api-versions v1/Endpoints --helm-api-versions v1/Event --helm-api-versions v1/LimitRange --helm-api-versions v1/Namespace --helm-api-versions v1/Node --helm-api-versions v1/PersistentVolume --helm-api-versions v1/PersistentVolumeClaim --helm-api-versions v1/Pod --helm-api-versions v1/PodTemplate --helm-api-versions v1/ReplicationController --helm-api-versions v1/ResourceQuota --helm-api-versions v1/Secret --helm-api-versions v1/Service --helm-api-versions v1/ServiceAccount --helm-api-versions velero.io/v1 --helm-api-versions velero.io/v1/Backup --helm-api-versions velero.io/v1/BackupRepository --helm-api-versions velero.io/v1/BackupStorageLocation --helm-api-versions velero.io/v1/DeleteBackupRequest --helm-api-versions velero.io/v1/DownloadRequest --helm-api-versions velero.io/v1/PodVolumeBackup --helm-api-versions velero.io/v1/PodVolumeRestore --helm-api-versions velero.io/v1/Restore --helm-api-versions velero.io/v1/Schedule --helm-api-versions velero.io/v1/ServerStatusRequest --helm-api-versions velero.io/v1/VolumeSnapshotLocation --helm-api-versions velero.io/v2alpha1 --helm-api-versions velero.io/v2alpha1/DataDownload --helm-api-versions velero.io/v2alpha1/DataUpload --helm-api-versions whereabouts.cni.cncf.io/v1alpha1 --helm-api-versions whereabouts.cni.cncf.io/v1alpha1/IPPool --helm-api-versions whereabouts.cni.cncf.io/v1alpha1/NodeSlicePool --helm-api-versions whereabouts.cni.cncf.io/v1alpha1/OverlappingRangeIPReservation --helm-api-versions work.open-cluster-management.io/v1 --helm-api-versions work.open-cluster-management.io/v1/AppliedManifestWork` failed exit status 1: Error: Error: failed to untar: a file or directory with the name /components/vector-kubearchive-log-collector/development/charts/loki-12.0.0/loki-12.0.0.tgz already exists [2026-05-09 22:51:33] [WARN] │ └─ : unable to run: 'helm pull --untar --untardir /components/vector-kubearchive-log-collector/development/charts/loki-12.0.0 --repo https://grafana-community.github.io/helm-charts loki --version 12.0.0' with env=[HELM_CONFIG_HOME=/tmp/kustomize-helm-1036363584/helm HELM_CACHE_HOME=/tmp/kustomize-helm-1036363584/helm/.cache HELM_DATA_HOME=/tmp/kustomize-helm-1036363584/helm/.data] (is 'helm' installed?): exit status 1 [2026-05-09 22:51:33] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 22:51:43] [PROGRESS] Applications: 29/44 ready | 15 pending (0m 31s elapsed) [2026-05-09 22:51:43] [INFO] Pending: application-api-in-cluster-local,disaster-recovery-in-cluster-local,enterprise-contract-in-cluster-local,integration-in-cluster-local,internal-services-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,squid-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-05-09 22:51:43] [WARN] Found applications in Unknown state (not Progressing), investigating... [2026-05-09 22:51:44] [ERROR] Application 'vector-kubearchive-log-collector-in-cluster-local' is in Unknown state without 'context deadline exceeded' Switched to branch 'main' Your branch is up to date with 'upstream/main'. I0509 22:51:44.235257 26061 common.go:278] got an error: exit status 141 - will retry in 10s [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed. Detected at: > goroutine 121 [running]: > runtime/debug.Stack() > /usr/lib/golang/src/runtime/debug/stack.go:26 +0x5e > sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot() > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/log/log.go:60 +0xcd > sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).WithName(0xc00001b3c0, {0x3566ea2, 0x14}) > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/log/deleg.go:147 +0x3e > github.com/go-logr/logr.Logger.WithName({{0x3c82b38, 0xc00001b3c0}, 0x0}, {0x3566ea2?, 0x0?}) > /opt/app-root/src/go/pkg/mod/github.com/go-logr/logr@v1.4.3/logr.go:345 +0x36 > sigs.k8s.io/controller-runtime/pkg/client.newClient(0x1?, {0x0, 0xc0007fddc0, {0x0, 0x0}, 0x0, 0x0}) > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/client/client.go:118 +0xdb > sigs.k8s.io/controller-runtime/pkg/client.New(0xc0007ce008?, {0x0, 0xc0007fddc0, {0x0, 0x0}, 0x0, 0x0}) > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/client/client.go:98 +0x4d > github.com/konflux-ci/e2e-tests/pkg/clients/kubernetes.NewAdminKubernetesClient() > /tmp/tmp.Fe5UbAZrhJ/pkg/clients/kubernetes/client.go:159 +0x8f > github.com/konflux-ci/e2e-tests/magefiles/installation.NewAppStudioInstallController() > /tmp/tmp.Fe5UbAZrhJ/magefiles/installation/install.go:105 +0x31 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine/repos.InstallKonflux() > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/repos/common.go:262 +0x13 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine/repos.retry(0x38dc220, 0x2, 0x2540be400) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/repos/common.go:281 +0xf9 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine/repos.init.func7(0x3c3fac0?) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/repos/common.go:355 +0xae > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.ActionFunc.Execute(0xc?, 0x35414f0?) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:279 +0x19 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Apply(...) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:315 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Check(0x5c46f80, 0xc0005f5908) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:348 +0xb3 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.All.Check({0x5c3f320?, 0xc0012d7bd8?, 0x24207f9?}, 0xc0005f5908) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:245 +0x4f > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Eval(...) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:308 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Check(0x5c470a0, 0xc0005f5908) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:340 +0x2b > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.All.Check({0x5c48d20?, 0x2?, 0xc000028540?}, 0xc0005f5908) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:245 +0x4f > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Eval(...) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:308 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*RuleEngine).runLoadedCatalog(0x5c77e90, {0xc000a8a508?, 0xc000c1de60?, 0x3591cbf?}, 0xc0005f5908) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:129 +0x11f > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*RuleEngine).RunRulesOfCategory(0x5c77e90, {0x353b682, 0x2}, 0xc0005f5908) > /tmp/tmp.Fe5UbAZrhJ/magefiles/rulesengine/types.go:121 +0x1b4 > main.CI.TestE2E({}) > /tmp/tmp.Fe5UbAZrhJ/magefiles/magefile.go:330 +0x18a > main.main.func20({0x0?, 0x0?}) > /tmp/tmp.Fe5UbAZrhJ/magefiles/mage_output_file.go:888 +0xf > main.main.func12.1() > /tmp/tmp.Fe5UbAZrhJ/magefiles/mage_output_file.go:306 +0x5b > created by main.main.func12 in goroutine 1 > /tmp/tmp.Fe5UbAZrhJ/magefiles/mage_output_file.go:301 +0xbe W0509 22:51:54.237174 26061 install.go:185] folder /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments already exists... removing I0509 22:51:54.371306 26061 install.go:226] cloning 'https://github.com/redhat-appstudio/infra-deployments' (ref: 'refs/heads/main', remote: 'upstream') Enumerating objects: 80723, done. Counting objects: 3% (1/31) Counting objects: 6% (2/31) Counting objects: 9% (3/31) Counting objects: 12% (4/31) Counting objects: 16% (5/31) Counting objects: 19% (6/31) Counting objects: 22% (7/31) Counting objects: 25% (8/31) Counting objects: 29% (9/31) Counting objects: 32% (10/31) Counting objects: 35% (11/31) Counting objects: 38% (12/31) Counting objects: 41% (13/31) Counting objects: 45% (14/31) Counting objects: 48% (15/31) Counting objects: 51% (16/31) Counting objects: 54% (17/31) Counting objects: 58% (18/31) Counting objects: 61% (19/31) Counting objects: 64% (20/31) Counting objects: 67% (21/31) Counting objects: 70% (22/31) Counting objects: 74% (23/31) Counting objects: 77% (24/31) Counting objects: 80% (25/31) Counting objects: 83% (26/31) Counting objects: 87% (27/31) Counting objects: 90% (28/31) Counting objects: 93% (29/31) Counting objects: 96% (30/31) Counting objects: 100% (31/31) Counting objects: 100% (31/31), done. Compressing objects: 3% (1/29) Compressing objects: 6% (2/29) Compressing objects: 10% (3/29) Compressing objects: 13% (4/29) Compressing objects: 17% (5/29) Compressing objects: 20% (6/29) Compressing objects: 24% (7/29) Compressing objects: 27% (8/29) Compressing objects: 31% (9/29) Compressing objects: 34% (10/29) Compressing objects: 37% (11/29) Compressing objects: 41% (12/29) Compressing objects: 44% (13/29) Compressing objects: 48% (14/29) Compressing objects: 51% (15/29) Compressing objects: 55% (16/29) Compressing objects: 58% (17/29) Compressing objects: 62% (18/29) Compressing objects: 65% (19/29) Compressing objects: 68% (20/29) Compressing objects: 72% (21/29) Compressing objects: 75% (22/29) Compressing objects: 79% (23/29) Compressing objects: 82% (24/29) Compressing objects: 86% (25/29) Compressing objects: 89% (26/29) Compressing objects: 93% (27/29) Compressing objects: 96% (28/29) Compressing objects: 100% (29/29) Compressing objects: 100% (29/29), done. Total 80723 (delta 11), reused 0 (delta 0), pack-reused 80692 (from 2) From https://github.com/redhat-appstudio/infra-deployments * branch main -> FETCH_HEAD Already up to date. ============================================================================= [2026-05-09 22:51:58] [STEP] Starting Konflux Cluster Bootstrap ============================================================================= [2026-05-09 22:51:58] [INFO] Mode: preview [2026-05-09 22:51:58] [INFO] Options: OBO=disabled, EAAS=disabled [2026-05-09 22:51:58] [INFO] Start time: 2026-05-09 22:51:58 UTC ============================================================================= [2026-05-09 22:51:58] [STEP] Phase 1: Deploying ArgoCD ============================================================================= ============================================================================= [2026-05-09 22:51:58] [STEP] Deploying OpenShift GitOps (ArgoCD) ============================================================================= [2026-05-09 22:51:58] [SUBSTEP] Verifying cluster-admin permissions [2026-05-09 22:51:58] [SUCCESS] User 'cluster-admin' has cluster-admin permissions [2026-05-09 22:51:58] [SUBSTEP] Installing OpenShift GitOps operator subscription clusterrole.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-application-controller unchanged clusterrole.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-server unchanged clusterrolebinding.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-application-controller unchanged clusterrolebinding.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-server unchanged subscription.operators.coreos.com/openshift-gitops-operator unchanged [2026-05-09 22:52:00] [SUCCESS] GitOps operator subscription applied [2026-05-09 22:52:00] [SUBSTEP] Waiting for default ArgoCD project to be created [2026-05-09 22:52:00] [SUCCESS] Default ArgoCD project is available [2026-05-09 22:52:00] [SUBSTEP] Waiting for OpenShift GitOps route to be created [2026-05-09 22:52:00] [SUCCESS] OpenShift GitOps route is available [2026-05-09 22:52:00] [SUBSTEP] Configuring ArgoCD repo server resources and timeout argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:00] [SUCCESS] Repo server configured: timeout=5m, cpu=100m, memory=100Mi [2026-05-09 22:52:00] [SUBSTEP] Configuring ArgoCD application controller resources argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:00] [SUCCESS] Application controller configured: cpu=4, memory=4Gi [2026-05-09 22:52:00] [SUBSTEP] Switching ArgoCD route to re-encryption TLS argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:00] [SUCCESS] Route TLS termination set to 'reencrypt' [2026-05-09 22:52:00] [SUBSTEP] Restarting ArgoCD server after TLS configuration change pod "openshift-gitops-server-588f67d9fb-9zrnh" deleted [2026-05-09 22:52:01] [SUCCESS] ArgoCD server pods deleted for restart [2026-05-09 22:52:01] [SUBSTEP] Granting admin role to authenticated users [2026-05-09 22:52:01] [INFO] Note: This should be updated once proper access policy is in place argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:01] [SUCCESS] RBAC policy set: system:authenticated -> role:admin [2026-05-09 22:52:01] [SUBSTEP] Configuring PVC health check (WaitForFirstConsumer workaround) Warning: unknown field "spec.resourceCustomizations" argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:02] [SUCCESS] PVC health customization applied (Pending/Bound = Healthy) [2026-05-09 22:52:02] [SUBSTEP] Enabling Helm support in Kustomize builds argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:02] [SUCCESS] Kustomize build options set: --enable-helm [2026-05-09 22:52:02] [SUBSTEP] Configuring ArgoCD to ignore aggregated roles in diff argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:02] [SUCCESS] ignoreAggregatedRoles set to true [2026-05-09 22:52:02] [SUBSTEP] Setting ArgoCD tracking method to annotation argocd.argoproj.io/openshift-gitops patched (no change) [2026-05-09 22:52:02] [SUCCESS] Resource tracking method set to 'annotation' [2026-05-09 22:52:02] [SUBSTEP] Restarting GitOps server deployment deployment.apps/openshift-gitops-server restarted [2026-05-09 22:52:02] [SUCCESS] GitOps server restart initiated [2026-05-09 22:52:02] [INFO] Waiting for rollout to complete... Waiting for deployment "openshift-gitops-server" rollout to finish: 0 out of 1 new replicas have been updated... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... deployment "openshift-gitops-server" successfully rolled out [2026-05-09 22:52:33] [SUCCESS] GitOps server rollout complete ============================================================================= [2026-05-09 22:52:33] [STEP] ArgoCD Access Information ============================================================================= [2026-05-09 22:52:33] [INFO] ArgoCD URL: https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:52:33] [INFO] Authentication: Use 'Login with OpenShift' button (OpenShift credentials) [2026-05-09 22:52:33] [SUBSTEP] Verifying ArgoCD route is accessible [2026-05-09 22:52:53] [WAITING] Waiting for route to respond (5s/120s) [2026-05-09 22:52:58] [SUCCESS] ArgoCD is accessible at https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [ARGOCD_DEPLOY_JSON] {"status":"success","url":"https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com","namespace":"openshift-gitops","auth_method":"openshift"} ============================================================================= [2026-05-09 22:52:58] [STEP] OpenShift GitOps Deployment Complete ============================================================================= ============================================================================= [2026-05-09 22:52:58] [STEP] Phase 2: Bootstrapping Host Cluster ============================================================================= [2026-05-09 22:52:58] [INFO] Bootstrapping host cluster components [2026-05-09 22:52:58] [INFO] No preview.env file found - using environment variables only [2026-05-09 22:52:58] [SUBSTEP] Configuring Quality Dashboard secrets Setting secrets for Quality Dashboard namespace/quality-dashboard configured [2026-05-09 22:52:59] [SUCCESS] Quality Dashboard secrets configured [2026-05-09 22:52:59] [SUBSTEP] Configuring CI Helper App secret Creating secret for CI Helper App namespace/ci-helper-app configured [2026-05-09 22:53:00] [SUCCESS] CI Helper App secret configured [2026-05-09 22:53:00] [SUCCESS] Host cluster bootstrap complete ============================================================================= [2026-05-09 22:53:00] [STEP] Phase 3: Bootstrapping Member Cluster ============================================================================= [2026-05-09 22:53:00] [INFO] Bootstrapping member cluster components [2026-05-09 22:53:00] [INFO] No preview.env file found - using environment variables only [2026-05-09 22:53:00] [SUBSTEP] Configuring Pipeline Service secrets Setting secrets for pipeline-service tekton-results namespace already exists, skipping creation tekton-logging namespace already exists, skipping creation product-kubearchive-logging namespace already exists, skipping creation Creating DB secret DB secret already exists, skipping creation Creating S3 secret S3 secret already exists, skipping creation Creating S3 secret S3 secret already exists, skipping creation Creating Postgres TLS certs Postgres DB cert secret already exists, skipping creation [2026-05-09 22:53:01] [SUCCESS] Pipeline Service secrets configured [2026-05-09 22:53:01] [SUBSTEP] Configuring GitHub secrets [2026-05-09 22:53:01] [INFO] GitHub token provided - creating secrets namespace/application-service configured Creating a has secret from legacy token secret/has-github-token configured [2026-05-09 22:53:02] [SUCCESS] GitHub secrets configured [2026-05-09 22:53:02] [SUBSTEP] Configuring Image Controller secrets [2026-05-09 22:53:02] [INFO] Quay organization: redhat-appstudio-qe Creating a secret with a token for Image Controller namespace/image-controller configured secret/quaytoken configured [2026-05-09 22:53:03] [SUCCESS] Image Controller secrets configured [2026-05-09 22:53:03] [SUCCESS] Member cluster bootstrap complete ============================================================================= [2026-05-09 22:53:03] [STEP] Phase 4: Bootstrapping Common Components ============================================================================= [2026-05-09 22:53:03] [INFO] Bootstrapping common cluster components [2026-05-09 22:53:03] [INFO] No preview.env file found at /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../hack/preview.env - using environment variables only [2026-05-09 22:53:03] [SUBSTEP] Configuring Docker Hub pull secret Configuring the cluster with a pull secret for Docker Hub Saved credentials for docker.io into /tmp/tmp.tvlpNKMl8L secret/pull-secret data updated Saved credentials for docker.io into /tmp/tmp.tvlpNKMl8L secret/docker-io-pull configured [2026-05-09 22:53:05] [SUCCESS] Docker Hub pull secret configured [2026-05-09 22:53:05] [SUBSTEP] Configuring DORA metrics exporter secrets Setting secrets for Dora metrics exporter namespace/dora-metrics configured [2026-05-09 22:53:05] [SUCCESS] DORA metrics exporter secrets configured [2026-05-09 22:53:05] [SUCCESS] Common cluster bootstrap complete ============================================================================= [2026-05-09 22:53:05] [STEP] Phase 5: Mode-Specific Deployment (preview) ============================================================================= [2026-05-09 22:53:05] [INFO] Deploying preview configuration ============================================================================= [2026-05-09 22:53:05] [STEP] Starting Konflux Preview Environment Setup ============================================================================= [2026-05-09 22:53:05] [INFO] Script: /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../hack/preview.sh [2026-05-09 22:53:05] [INFO] Options: OBO=false, GRAFANA=false, EAAS=false [2026-05-09 22:53:05] [INFO] Start time: 2026-05-09 22:53:05 UTC ============================================================================= [2026-05-09 22:53:05] [STEP] Cluster Context Information ============================================================================= [2026-05-09 22:53:06] [INFO] OpenShift Version: 4.18.9 [2026-05-09 22:53:06] [INFO] API Server: https://api.kx-4ccd113921.l5qz.p3.openshiftapps.com:443 [2026-05-09 22:53:06] [INFO] Cluster ID: 04b7f83c-45d4-432b-b66f-7f0cf55568d2 [2026-05-09 22:53:06] [INFO] Cluster Domain: apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:53:06] [INFO] Total Nodes: 3 [2026-05-09 22:53:06] [INFO] - Master nodes: 0 [2026-05-09 22:53:06] [INFO] - Worker nodes: 3 [2026-05-09 22:53:06] [SUCCESS] Cluster operators: All healthy [CLUSTER_CONTEXT_JSON] {"ocp_version":"4.18.9","api_server":"https://api.kx-4ccd113921.l5qz.p3.openshiftapps.com:443","cluster_id":"04b7f83c-45d4-432b-b66f-7f0cf55568d2","cluster_domain":"apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com","total_nodes":3,"master_nodes":0,"worker_nodes":3,"degraded_operators":0} Switched to a new branch 'preview-main-ysxt' [2026-05-09 22:53:07] [SUCCESS] Git environment initialized [2026-05-09 22:53:07] [INFO] - Repository URL: https://github.com/redhat-appstudio-qe/infra-deployments.git [2026-05-09 22:53:07] [INFO] - Source branch: main [2026-05-09 22:53:07] [INFO] - Preview branch: preview-main-ysxt [2026-05-09 22:53:07] [INFO] - GitHub org: redhat-appstudio-qe ============================================================================= [2026-05-09 22:53:07] [STEP] Patching ArgoCD application manifests to use fork repository ============================================================================= [2026-05-09 22:53:07] [INFO] Setting repoURL to: https://github.com/redhat-appstudio-qe/infra-deployments.git [2026-05-09 22:53:07] [INFO] Setting targetRevision to: preview-main-ysxt [2026-05-09 22:53:07] [SUBSTEP] Patched: application-patch.yaml [2026-05-09 22:53:07] [SUBSTEP] Patched: application-set-patch.yaml [2026-05-09 22:53:07] [SUBSTEP] Patched: application-set-multisrc-src-1-patch.yaml [2026-05-09 22:53:07] [SUCCESS] All ArgoCD patch files updated ============================================================================= [2026-05-09 22:53:07] [STEP] Labeling cluster nodes for Konflux workloads ============================================================================= [2026-05-09 22:53:07] [INFO] Reference: https://github.com/redhat-appstudio/infra-deployments/pull/4415 [2026-05-09 22:53:07] [INFO] Found 3 nodes to label with 'konflux-ci.dev/workload=konflux-tenants' [2026-05-09 22:53:07] [SUBSTEP] Labeling node/ip-10-0-137-137.ec2.internal node/ip-10-0-137-137.ec2.internal not labeled [2026-05-09 22:53:07] [SUCCESS] Successfully labeled node/ip-10-0-137-137.ec2.internal [2026-05-09 22:53:07] [SUBSTEP] Labeling node/ip-10-0-157-249.ec2.internal node/ip-10-0-157-249.ec2.internal not labeled [2026-05-09 22:53:07] [SUCCESS] Successfully labeled node/ip-10-0-157-249.ec2.internal [2026-05-09 22:53:07] [SUBSTEP] Labeling node/ip-10-0-169-212.ec2.internal node/ip-10-0-169-212.ec2.internal not labeled [2026-05-09 22:53:07] [SUCCESS] Successfully labeled node/ip-10-0-169-212.ec2.internal [2026-05-09 22:53:07] [SUBSTEP] Verifying labels on all nodes... [2026-05-09 22:53:07] [SUCCESS] All 3 nodes labeled and verified successfully ============================================================================= [2026-05-09 22:53:07] [STEP] Checking OCP version for Kueue compatibility ============================================================================= [2026-05-09 22:53:07] [INFO] Detected OpenShift Container Platform version: 4.18.9 (minor: 18) [2026-05-09 22:53:07] [SUCCESS] OCP version 4.18.9 meets Kueue requirements - Kueue will be deployed ============================================================================= [2026-05-09 22:53:07] [STEP] Configuring GitHub organization ============================================================================= [2026-05-09 22:53:07] [INFO] Setting GitHub org to: redhat-appstudio-qe [2026-05-09 22:53:07] [INFO] Configuring GitHub organization for Application Service (HAS) [2026-05-09 22:53:08] [INFO] - Target org: redhat-appstudio-qe [2026-05-09 22:53:08] [INFO] - Config file: /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../components/has/base/kustomization.yaml [2026-05-09 22:53:08] [SUCCESS] GitHub organization set to 'redhat-appstudio-qe' [2026-05-09 22:53:08] [SUCCESS] Verified: Configuration updated correctly [2026-05-09 22:53:08] [SUCCESS] GitHub organization configured ============================================================================= [2026-05-09 22:53:08] [STEP] Configuring Rekor server hostname ============================================================================= [2026-05-09 22:53:08] [INFO] Cluster domain: apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:53:08] [INFO] Rekor server hostname: rekor.apps.rosa.kx-4ccd113921.l5qz.p3.openshiftapps.com [2026-05-09 22:53:08] [SUCCESS] Rekor server hostname configured ============================================================================= [2026-05-09 22:53:08] [STEP] Applying service image overrides from environment variables ============================================================================= [2026-05-09 22:53:08] [INFO] No service image overrides configured - using default images ============================================================================= [2026-05-09 22:53:08] [STEP] Committing and pushing preview changes ============================================================================= [preview-main-ysxt 20934f81e] Preview mode, do not merge into main 8 files changed, 15 insertions(+), 19 deletions(-) remote: remote: Create a pull request for 'preview-main-ysxt' on GitHub by visiting: remote: https://github.com/redhat-appstudio-qe/infra-deployments/pull/new/preview-main-ysxt remote: To https://github.com/redhat-appstudio-qe/infra-deployments.git * [new branch] preview-main-ysxt -> preview-main-ysxt branch 'preview-main-ysxt' set up to track 'qe/preview-main-ysxt'. [2026-05-09 22:53:09] [SUCCESS] Preview changes committed and pushed to qe/preview-main-ysxt ============================================================================= [2026-05-09 22:53:09] [STEP] Deploying ArgoCD applications ============================================================================= [2026-05-09 22:53:09] [SUBSTEP] Applying root Application from: /tmp/tmp.Fe5UbAZrhJ/tmp/infra-deployments/hack/../argo-cd-apps/app-of-app-sets/development application.argoproj.io/all-application-sets configured [2026-05-09 22:53:09] [SUCCESS] Root Application 'all-application-sets' created [2026-05-09 22:53:09] [SUBSTEP] Waiting for 'all-application-sets' to become Healthy and Synced [2026-05-09 22:53:09] [SUCCESS] Root application 'all-application-sets' is Healthy and Synced [2026-05-09 22:53:09] [SUBSTEP] Triggering hard refresh on all ArgoCD applications [2026-05-09 22:53:10] [INFO] Found 44 applications to refresh application.argoproj.io/disaster-recovery-in-cluster-local patched application.argoproj.io/build-service-in-cluster-local patched application.argoproj.io/build-templates-in-cluster-local patched application.argoproj.io/has-in-cluster-local patched application.argoproj.io/vector-kubearchive-log-collector-in-cluster-local patched application.argoproj.io/konflux-rbac-in-cluster-local patched application.argoproj.io/repository-validator-in-cluster-local patched application.argoproj.io/backup-in-cluster-local patched application.argoproj.io/image-rbac-proxy-in-cluster-local patched application.argoproj.io/project-controller-in-cluster-local patched application.argoproj.io/mintmaker-in-cluster-local patched application.argoproj.io/kubearchive-in-cluster-local patched application.argoproj.io/crossplane-control-plane-in-cluster-local patched application.argoproj.io/image-controller-in-cluster-local patched application.argoproj.io/kueue-in-cluster-local patched application.argoproj.io/smee-client-in-cluster-local patched application.argoproj.io/all-application-sets patched application.argoproj.io/policies-in-cluster-local patched application.argoproj.io/tempo-in-cluster-local patched application.argoproj.io/trust-manager-in-cluster-local patched application.argoproj.io/enterprise-contract-in-cluster-local patched application.argoproj.io/postgres patched application.argoproj.io/dora-metrics-in-cluster-local patched application.argoproj.io/monitoring-registry-in-cluster-local patched application.argoproj.io/konflux-kite-in-cluster-local patched application.argoproj.io/vector-tekton-logs-collector-in-cluster-local patched application.argoproj.io/release-in-cluster-local patched application.argoproj.io/internal-services-in-cluster-local patched application.argoproj.io/monitoring-cardinality-in-cluster-local patched application.argoproj.io/tracing-workload-tracing-in-cluster-local patched application.argoproj.io/kyverno-in-cluster-local patched application.argoproj.io/integration-in-cluster-local patched application.argoproj.io/cert-manager-in-cluster-local patched application.argoproj.io/application-api-in-cluster-local patched application.argoproj.io/knative-eventing-in-cluster-local patched application.argoproj.io/squid-in-cluster-local patched application.argoproj.io/perf-team-prometheus-reader-in-cluster-local patched application.argoproj.io/disable-csvcopy-in-cluster-local patched application.argoproj.io/pipeline-service-in-cluster-local patched application.argoproj.io/monitoring-workload-custom-kube-state-metrics-in-cluster-local patched application.argoproj.io/konflux-info-in-cluster-local patched application.argoproj.io/tracing-workload-otel-collector-in-cluster-local patched application.argoproj.io/multi-platform-controller-in-cluster-local patched application.argoproj.io/monitoring-workload-prometheus-in-cluster-local patched [2026-05-09 22:53:19] [SUCCESS] Hard refresh triggered on all 44 applications [2026-05-09 22:53:19] [SUBSTEP] Waiting for refresh operations to complete [2026-05-09 22:53:19] [PROGRESS] Refresh: 7/44 complete | 37 still refreshing (5s elapsed) [2026-05-09 22:53:25] [PROGRESS] Refresh: 11/44 complete | 33 still refreshing (10s elapsed) [2026-05-09 22:53:30] [PROGRESS] Refresh: 28/44 complete | 16 still refreshing (15s elapsed) [2026-05-09 22:53:35] [PROGRESS] Refresh: 35/44 complete | 9 still refreshing (20s elapsed) [2026-05-09 22:53:41] [PROGRESS] Refresh: 40/44 complete | 4 still refreshing (25s elapsed) [2026-05-09 22:53:46] [PROGRESS] Refresh: 40/44 complete | 4 still refreshing (30s elapsed) [2026-05-09 22:53:51] [PROGRESS] Refresh: 42/44 complete | 2 still refreshing (35s elapsed) [2026-05-09 22:53:57] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (40s elapsed) [2026-05-09 22:54:02] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (45s elapsed) [2026-05-09 22:54:07] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (50s elapsed) [2026-05-09 22:54:13] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (55s elapsed) [2026-05-09 22:54:18] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (60s elapsed) [2026-05-09 22:54:23] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (65s elapsed) [2026-05-09 22:54:29] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (70s elapsed) [2026-05-09 22:54:34] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (75s elapsed) [2026-05-09 22:54:39] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (80s elapsed) [2026-05-09 22:54:45] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (85s elapsed) [2026-05-09 22:54:50] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (90s elapsed) [2026-05-09 22:54:55] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (95s elapsed) [2026-05-09 22:55:01] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (100s elapsed) [2026-05-09 22:55:06] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (105s elapsed) [2026-05-09 22:55:11] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (110s elapsed) [2026-05-09 22:55:17] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (115s elapsed) [2026-05-09 22:55:22] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (120s elapsed) [2026-05-09 22:55:27] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (125s elapsed) [2026-05-09 22:55:33] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (130s elapsed) [2026-05-09 22:55:38] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (135s elapsed) [2026-05-09 22:55:43] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (140s elapsed) [2026-05-09 22:55:49] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (145s elapsed) [2026-05-09 22:55:54] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (150s elapsed) [2026-05-09 22:55:59] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (155s elapsed) [2026-05-09 22:56:05] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (160s elapsed) [2026-05-09 22:56:10] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (165s elapsed) [2026-05-09 22:56:15] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (170s elapsed) [2026-05-09 22:56:21] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (175s elapsed) [2026-05-09 22:56:26] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (180s elapsed) [2026-05-09 22:56:31] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (185s elapsed) [2026-05-09 22:56:37] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (190s elapsed) [2026-05-09 22:56:42] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (195s elapsed) [2026-05-09 22:56:47] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (200s elapsed) [2026-05-09 22:56:53] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (205s elapsed) [2026-05-09 22:56:58] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (210s elapsed) [2026-05-09 22:57:03] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (215s elapsed) [2026-05-09 22:57:09] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (220s elapsed) [2026-05-09 22:57:14] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (225s elapsed) [2026-05-09 22:57:19] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (230s elapsed) [2026-05-09 22:57:25] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (235s elapsed) [2026-05-09 22:57:30] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (240s elapsed) [2026-05-09 22:57:35] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (245s elapsed) [2026-05-09 22:57:41] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (250s elapsed) [2026-05-09 22:57:46] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (255s elapsed) [2026-05-09 22:57:51] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (260s elapsed) [2026-05-09 22:57:57] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (265s elapsed) [2026-05-09 22:58:02] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (270s elapsed) [2026-05-09 22:58:07] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (275s elapsed) [2026-05-09 22:58:13] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (280s elapsed) [2026-05-09 22:58:18] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (285s elapsed) [2026-05-09 22:58:23] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (290s elapsed) [2026-05-09 22:58:29] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (295s elapsed) [2026-05-09 22:58:34] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (300s elapsed) [2026-05-09 22:58:39] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (305s elapsed) [2026-05-09 22:58:45] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (310s elapsed) [2026-05-09 22:58:50] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (315s elapsed) [2026-05-09 22:58:55] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (320s elapsed) [2026-05-09 22:59:01] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (325s elapsed) [2026-05-09 22:59:06] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (330s elapsed) [2026-05-09 22:59:11] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (335s elapsed) [2026-05-09 22:59:17] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (340s elapsed) [2026-05-09 22:59:22] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (345s elapsed) [2026-05-09 22:59:27] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (350s elapsed) [2026-05-09 22:59:33] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (355s elapsed) [2026-05-09 22:59:38] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (360s elapsed) [2026-05-09 22:59:43] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (365s elapsed) [2026-05-09 22:59:49] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (370s elapsed) [2026-05-09 22:59:54] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (375s elapsed) [2026-05-09 22:59:59] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (380s elapsed) [2026-05-09 23:00:05] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (385s elapsed) [2026-05-09 23:00:10] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (390s elapsed) [2026-05-09 23:00:15] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (395s elapsed) [2026-05-09 23:00:21] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (400s elapsed) [2026-05-09 23:00:26] [PROGRESS] Refresh: 43/44 complete | 1 still refreshing (405s elapsed) [2026-05-09 23:00:31] [SUCCESS] All 44 applications refreshed ============================================================================= [2026-05-09 23:00:31] [STEP] Waiting for all ArgoCD applications to sync and become healthy ============================================================================= [2026-05-09 23:00:31] [INFO] Timeout: 2700 seconds (45 minutes) [2026-05-09 23:00:32] [PROGRESS] Applications: 43/44 ready | 1 pending (0m 0s elapsed) [2026-05-09 23:00:32] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:00:32] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:00:42] [PROGRESS] Applications: 43/44 ready | 1 pending (0m 11s elapsed) [2026-05-09 23:00:42] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:00:42] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:00:52] [PROGRESS] Applications: 43/44 ready | 1 pending (0m 21s elapsed) [2026-05-09 23:00:52] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:00:52] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:01:02] [PROGRESS] Applications: 43/44 ready | 1 pending (0m 31s elapsed) [2026-05-09 23:01:02] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:01:02] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:01:12] [PROGRESS] Applications: 43/44 ready | 1 pending (0m 41s elapsed) [2026-05-09 23:01:12] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:01:12] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:01:23] [PROGRESS] Applications: 43/44 ready | 1 pending (0m 51s elapsed) [2026-05-09 23:01:23] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:01:23] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:01:33] [PROGRESS] Applications: 43/44 ready | 1 pending (1m 2s elapsed) [2026-05-09 23:01:33] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:01:33] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:01:43] [PROGRESS] Applications: 43/44 ready | 1 pending (1m 12s elapsed) [2026-05-09 23:01:43] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:01:43] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:01:53] [PROGRESS] Applications: 43/44 ready | 1 pending (1m 22s elapsed) [2026-05-09 23:01:53] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:01:53] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:02:03] [PROGRESS] Applications: 43/44 ready | 1 pending (1m 32s elapsed) [2026-05-09 23:02:03] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:02:03] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:02:14] [PROGRESS] Applications: 43/44 ready | 1 pending (1m 42s elapsed) [2026-05-09 23:02:14] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:02:14] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:02:24] [PROGRESS] Applications: 43/44 ready | 1 pending (1m 53s elapsed) [2026-05-09 23:02:24] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:02:24] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:02:34] [PROGRESS] Applications: 43/44 ready | 1 pending (2m 3s elapsed) [2026-05-09 23:02:34] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:02:34] [SUBSTEP] Detailed status of pending applications: [2026-05-09 23:02:34] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-05-09 23:02:34] [INFO] │ ├─ Sync Status: OutOfSync [2026-05-09 23:02:34] [INFO] │ ├─ Health Status: Healthy [2026-05-09 23:02:34] [INFO] │ ├─ Out-of-sync resources: 1 [2026-05-09 23:02:34] [INFO] │ └─ Message: No message [2026-05-09 23:02:34] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:02:44] [PROGRESS] Applications: 43/44 ready | 1 pending (2m 13s elapsed) [2026-05-09 23:02:44] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:02:44] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:02:54] [PROGRESS] Applications: 43/44 ready | 1 pending (2m 23s elapsed) [2026-05-09 23:02:54] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:02:54] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:03:05] [PROGRESS] Applications: 43/44 ready | 1 pending (2m 33s elapsed) [2026-05-09 23:03:05] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:03:05] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:03:15] [PROGRESS] Applications: 43/44 ready | 1 pending (2m 44s elapsed) [2026-05-09 23:03:15] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:03:15] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:03:25] [PROGRESS] Applications: 43/44 ready | 1 pending (2m 54s elapsed) [2026-05-09 23:03:25] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:03:25] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:03:35] [PROGRESS] Applications: 43/44 ready | 1 pending (3m 4s elapsed) [2026-05-09 23:03:35] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:03:35] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:03:45] [PROGRESS] Applications: 43/44 ready | 1 pending (3m 14s elapsed) [2026-05-09 23:03:45] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:03:45] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:03:56] [PROGRESS] Applications: 43/44 ready | 1 pending (3m 24s elapsed) [2026-05-09 23:03:56] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:03:56] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:04:06] [PROGRESS] Applications: 43/44 ready | 1 pending (3m 35s elapsed) [2026-05-09 23:04:06] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:04:06] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:04:16] [PROGRESS] Applications: 43/44 ready | 1 pending (3m 45s elapsed) [2026-05-09 23:04:16] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:04:16] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:04:26] [PROGRESS] Applications: 43/44 ready | 1 pending (3m 55s elapsed) [2026-05-09 23:04:26] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:04:26] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:04:36] [PROGRESS] Applications: 43/44 ready | 1 pending (4m 5s elapsed) [2026-05-09 23:04:36] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:04:36] [SUBSTEP] Detailed status of pending applications: [2026-05-09 23:04:37] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-05-09 23:04:37] [INFO] │ ├─ Sync Status: OutOfSync [2026-05-09 23:04:37] [INFO] │ ├─ Health Status: Healthy [2026-05-09 23:04:37] [INFO] │ ├─ Out-of-sync resources: 1 [2026-05-09 23:04:37] [INFO] │ └─ Message: No message [2026-05-09 23:04:37] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:04:47] [PROGRESS] Applications: 43/44 ready | 1 pending (4m 16s elapsed) [2026-05-09 23:04:47] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:04:47] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:04:57] [PROGRESS] Applications: 43/44 ready | 1 pending (4m 26s elapsed) [2026-05-09 23:04:57] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:04:57] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:05:07] [PROGRESS] Applications: 43/44 ready | 1 pending (4m 36s elapsed) [2026-05-09 23:05:07] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:05:07] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:05:17] [PROGRESS] Applications: 43/44 ready | 1 pending (4m 46s elapsed) [2026-05-09 23:05:17] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:05:17] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:05:27] [PROGRESS] Applications: 43/44 ready | 1 pending (4m 56s elapsed) [2026-05-09 23:05:27] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:05:27] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:05:38] [PROGRESS] Applications: 43/44 ready | 1 pending (5m 6s elapsed) [2026-05-09 23:05:38] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:05:38] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:05:48] [PROGRESS] Applications: 43/44 ready | 1 pending (5m 17s elapsed) [2026-05-09 23:05:48] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:05:48] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:05:58] [PROGRESS] Applications: 43/44 ready | 1 pending (5m 27s elapsed) [2026-05-09 23:05:58] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:05:58] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:06:08] [PROGRESS] Applications: 43/44 ready | 1 pending (5m 37s elapsed) [2026-05-09 23:06:08] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:06:08] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:06:18] [PROGRESS] Applications: 43/44 ready | 1 pending (5m 47s elapsed) [2026-05-09 23:06:18] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:06:18] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:06:29] [PROGRESS] Applications: 43/44 ready | 1 pending (5m 57s elapsed) [2026-05-09 23:06:29] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:06:29] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:06:39] [PROGRESS] Applications: 43/44 ready | 1 pending (6m 8s elapsed) [2026-05-09 23:06:39] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:06:39] [SUBSTEP] Detailed status of pending applications: [2026-05-09 23:06:39] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-05-09 23:06:39] [INFO] │ ├─ Sync Status: OutOfSync [2026-05-09 23:06:39] [INFO] │ ├─ Health Status: Healthy [2026-05-09 23:06:39] [INFO] │ ├─ Out-of-sync resources: 1 [2026-05-09 23:06:39] [INFO] │ └─ Message: No message [2026-05-09 23:06:39] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:06:49] [PROGRESS] Applications: 43/44 ready | 1 pending (6m 18s elapsed) [2026-05-09 23:06:49] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:06:49] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:06:59] [PROGRESS] Applications: 43/44 ready | 1 pending (6m 28s elapsed) [2026-05-09 23:06:59] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:06:59] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:07:10] [PROGRESS] Applications: 43/44 ready | 1 pending (6m 38s elapsed) [2026-05-09 23:07:10] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:07:10] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:07:20] [PROGRESS] Applications: 43/44 ready | 1 pending (6m 49s elapsed) [2026-05-09 23:07:20] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:07:20] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:07:30] [PROGRESS] Applications: 43/44 ready | 1 pending (6m 59s elapsed) [2026-05-09 23:07:30] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:07:30] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:07:40] [PROGRESS] Applications: 43/44 ready | 1 pending (7m 9s elapsed) [2026-05-09 23:07:40] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:07:40] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:07:50] [PROGRESS] Applications: 43/44 ready | 1 pending (7m 19s elapsed) [2026-05-09 23:07:50] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:07:50] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:08:01] [PROGRESS] Applications: 43/44 ready | 1 pending (7m 29s elapsed) [2026-05-09 23:08:01] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:08:01] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:08:11] [PROGRESS] Applications: 43/44 ready | 1 pending (7m 40s elapsed) [2026-05-09 23:08:11] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:08:11] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:08:21] [PROGRESS] Applications: 43/44 ready | 1 pending (7m 50s elapsed) [2026-05-09 23:08:21] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:08:21] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:08:31] [PROGRESS] Applications: 43/44 ready | 1 pending (8m 0s elapsed) [2026-05-09 23:08:31] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:08:31] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:08:41] [PROGRESS] Applications: 43/44 ready | 1 pending (8m 10s elapsed) [2026-05-09 23:08:41] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:08:41] [SUBSTEP] Detailed status of pending applications: [2026-05-09 23:08:42] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-05-09 23:08:42] [INFO] │ ├─ Sync Status: OutOfSync [2026-05-09 23:08:42] [INFO] │ ├─ Health Status: Healthy [2026-05-09 23:08:42] [INFO] │ ├─ Out-of-sync resources: 1 [2026-05-09 23:08:42] [INFO] │ └─ Message: No message [2026-05-09 23:08:42] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:08:52] [PROGRESS] Applications: 43/44 ready | 1 pending (8m 21s elapsed) [2026-05-09 23:08:52] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:08:52] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:09:02] [PROGRESS] Applications: 43/44 ready | 1 pending (8m 31s elapsed) [2026-05-09 23:09:02] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:09:02] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:09:12] [PROGRESS] Applications: 43/44 ready | 1 pending (8m 41s elapsed) [2026-05-09 23:09:12] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:09:12] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:09:22] [PROGRESS] Applications: 43/44 ready | 1 pending (8m 51s elapsed) [2026-05-09 23:09:22] [INFO] Pending: konflux-kite-in-cluster-local [2026-05-09 23:09:22] [WAITING] Waiting 10 seconds before next sync check... [2026-05-09 23:09:32] [PROGRESS] Applications: 42/44 ready | 2 pending (9m 1s elapsed) [2026-05-09 23:09:32] [INFO] Pending: konflux-kite-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-05-09 23:09:32] [WARN] Found applications in Unknown state (not Progressing), investigating... [2026-05-09 23:09:33] [ERROR] Application 'vector-kubearchive-log-collector-in-cluster-local' is in Unknown state without 'context deadline exceeded' Switched to branch 'main' Your branch is up to date with 'upstream/main'. Error: reached maximum number of attempts (2). error: exit status 141 make: *** [Makefile:25: ci/test/e2e] Error 1