./mage -v ci:teste2e Running target: CI:TestE2E I0421 17:31:34.756042 19172 magefile.go:526] setting up new custom bundle for testing... I0421 17:31:35.287790 19172 util.go:512] found credentials for image ref quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776792695-yeov -> user: redhat-appstudio-qe+redhat_appstudio_quality Creating Tekton Bundle: - Added Pipeline: docker-build to image I0421 17:31:36.519751 19172 bundle.go:57] image digest for a new tekton bundle quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776792695-yeov: quay.io/redhat-appstudio-qe/test-images@sha256:baca8fbb6fa2e3bf41c06f9a15d144eeb334f269ef62c450158bb94d51dde19a I0421 17:31:36.519780 19172 magefile.go:532] To use the custom docker bundle locally, run below cmd: export CUSTOM_DOCKER_BUILD_PIPELINE_BUNDLE=quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776792695-yeov I0421 17:31:36.519802 19172 e2e_repo.go:347] checking if repository is e2e-tests I0421 17:31:36.519811 19172 e2e_repo.go:335] multi-platform tests and require sprayproxy registering are set to TRUE exec: git "diff" "--name-status" "upstream/main..HEAD" I0421 17:31:36.522923 19172 util.go:451] The following files, tests/disaster-recovery/backup_infra.go, tests/disaster-recovery/const.go, tests/disaster-recovery/dr_backwards_compat.go, tests/disaster-recovery/dr_same_version.go, tests/disaster-recovery/tenant_application_lifecycle.go, were changed! exec: go "install" "-mod=mod" "github.com/onsi/ginkgo/v2/ginkgo" go: downloading github.com/go-task/slim-sprig/v3 v3.0.0 go: downloading github.com/google/pprof v0.0.0-20260106004452-d7df1bf2cac7 I0421 17:31:39.883409 19172 install.go:188] cloning 'https://github.com/redhat-appstudio/infra-deployments' with git ref 'refs/heads/main' Enumerating objects: 79412, done. Counting objects: 3% (1/29) Counting objects: 6% (2/29) Counting objects: 10% (3/29) Counting objects: 13% (4/29) Counting objects: 17% (5/29) Counting objects: 20% (6/29) Counting objects: 24% (7/29) Counting objects: 27% (8/29) Counting objects: 31% (9/29) Counting objects: 34% (10/29) Counting objects: 37% (11/29) Counting objects: 41% (12/29) Counting objects: 44% (13/29) Counting objects: 48% (14/29) Counting objects: 51% (15/29) Counting objects: 55% (16/29) Counting objects: 58% (17/29) Counting objects: 62% (18/29) Counting objects: 65% (19/29) Counting objects: 68% (20/29) Counting objects: 72% (21/29) Counting objects: 75% (22/29) Counting objects: 79% (23/29) Counting objects: 82% (24/29) Counting objects: 86% (25/29) Counting objects: 89% (26/29) Counting objects: 93% (27/29) Counting objects: 96% (28/29) Counting objects: 100% (29/29) Counting objects: 100% (29/29), done. Compressing objects: 3% (1/28) Compressing objects: 7% (2/28) Compressing objects: 10% (3/28) Compressing objects: 14% (4/28) Compressing objects: 17% (5/28) Compressing objects: 21% (6/28) Compressing objects: 25% (7/28) Compressing objects: 28% (8/28) Compressing objects: 32% (9/28) Compressing objects: 35% (10/28) Compressing objects: 39% (11/28) Compressing objects: 42% (12/28) Compressing objects: 46% (13/28) Compressing objects: 50% (14/28) Compressing objects: 53% (15/28) Compressing objects: 57% (16/28) Compressing objects: 60% (17/28) Compressing objects: 64% (18/28) Compressing objects: 67% (19/28) Compressing objects: 71% (20/28) Compressing objects: 75% (21/28) Compressing objects: 78% (22/28) Compressing objects: 82% (23/28) Compressing objects: 85% (24/28) Compressing objects: 89% (25/28) Compressing objects: 92% (26/28) Compressing objects: 96% (27/28) Compressing objects: 100% (28/28) Compressing objects: 100% (28/28), done. Total 79412 (delta 11), reused 1 (delta 0), pack-reused 79383 (from 1) From https://github.com/redhat-appstudio/infra-deployments * branch main -> FETCH_HEAD Already up to date. ============================================================================= [2026-04-21 17:31:43] [STEP] Starting Konflux Cluster Bootstrap ============================================================================= [2026-04-21 17:31:43] [INFO] Mode: preview [2026-04-21 17:31:43] [INFO] Options: OBO=disabled, EAAS=disabled [2026-04-21 17:31:43] [INFO] Start time: 2026-04-21 17:31:43 UTC ============================================================================= [2026-04-21 17:31:43] [STEP] Phase 1: Deploying ArgoCD ============================================================================= ============================================================================= [2026-04-21 17:31:43] [STEP] Deploying OpenShift GitOps (ArgoCD) ============================================================================= [2026-04-21 17:31:43] [SUBSTEP] Verifying cluster-admin permissions [2026-04-21 17:31:43] [SUCCESS] User 'cluster-admin' has cluster-admin permissions [2026-04-21 17:31:43] [SUBSTEP] Installing OpenShift GitOps operator subscription clusterrole.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-application-controller created clusterrole.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-server created clusterrolebinding.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-application-controller created clusterrolebinding.rbac.authorization.k8s.io/appstudio-openshift-gitops-argocd-server created subscription.operators.coreos.com/openshift-gitops-operator created [2026-04-21 17:31:43] [SUCCESS] GitOps operator subscription applied [2026-04-21 17:31:43] [SUBSTEP] Waiting for default ArgoCD project to be created [2026-04-21 17:31:44] [WAITING] Default project not yet available (5s/300s) [2026-04-21 17:31:49] [WAITING] Default project not yet available (10s/300s) [2026-04-21 17:31:54] [WAITING] Default project not yet available (15s/300s) [2026-04-21 17:31:59] [WAITING] Default project not yet available (20s/300s) [2026-04-21 17:32:05] [WAITING] Default project not yet available (25s/300s) [2026-04-21 17:32:10] [WAITING] Default project not yet available (30s/300s) [2026-04-21 17:32:15] [WAITING] Default project not yet available (35s/300s) [2026-04-21 17:32:20] [SUCCESS] Default ArgoCD project is available [2026-04-21 17:32:20] [SUBSTEP] Waiting for OpenShift GitOps route to be created [2026-04-21 17:32:20] [SUCCESS] OpenShift GitOps route is available [2026-04-21 17:32:20] [SUBSTEP] Configuring ArgoCD repo server resources and timeout argocd.argoproj.io/openshift-gitops patched [2026-04-21 17:32:20] [SUCCESS] Repo server configured: timeout=5m, cpu=100m, memory=100Mi [2026-04-21 17:32:20] [SUBSTEP] Configuring ArgoCD application controller resources argocd.argoproj.io/openshift-gitops patched [2026-04-21 17:32:21] [SUCCESS] Application controller configured: cpu=4, memory=4Gi [2026-04-21 17:32:21] [SUBSTEP] Switching ArgoCD route to re-encryption TLS argocd.argoproj.io/openshift-gitops patched [2026-04-21 17:32:21] [SUCCESS] Route TLS termination set to 'reencrypt' [2026-04-21 17:32:21] [SUBSTEP] Restarting ArgoCD server after TLS configuration change pod "openshift-gitops-server-6944966b75-sv4zr" deleted [2026-04-21 17:32:22] [SUCCESS] ArgoCD server pods deleted for restart [2026-04-21 17:32:22] [SUBSTEP] Granting admin role to authenticated users [2026-04-21 17:32:22] [INFO] Note: This should be updated once proper access policy is in place argocd.argoproj.io/openshift-gitops patched [2026-04-21 17:32:22] [SUCCESS] RBAC policy set: system:authenticated -> role:admin [2026-04-21 17:32:22] [SUBSTEP] Configuring PVC health check (WaitForFirstConsumer workaround) Warning: unknown field "spec.resourceCustomizations" argocd.argoproj.io/openshift-gitops patched (no change) [2026-04-21 17:32:22] [SUCCESS] PVC health customization applied (Pending/Bound = Healthy) [2026-04-21 17:32:22] [SUBSTEP] Enabling Helm support in Kustomize builds argocd.argoproj.io/openshift-gitops patched [2026-04-21 17:32:22] [SUCCESS] Kustomize build options set: --enable-helm [2026-04-21 17:32:22] [SUBSTEP] Configuring ArgoCD to ignore aggregated roles in diff argocd.argoproj.io/openshift-gitops patched [2026-04-21 17:32:23] [SUCCESS] ignoreAggregatedRoles set to true [2026-04-21 17:32:23] [SUBSTEP] Setting ArgoCD tracking method to annotation argocd.argoproj.io/openshift-gitops patched [2026-04-21 17:32:23] [SUCCESS] Resource tracking method set to 'annotation' [2026-04-21 17:32:23] [SUBSTEP] Restarting GitOps server deployment deployment.apps/openshift-gitops-server restarted [2026-04-21 17:32:23] [SUCCESS] GitOps server restart initiated [2026-04-21 17:32:23] [INFO] Waiting for rollout to complete... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "openshift-gitops-server" rollout to finish: 1 old replicas are pending termination... deployment "openshift-gitops-server" successfully rolled out [2026-04-21 17:32:53] [SUCCESS] GitOps server rollout complete ============================================================================= [2026-04-21 17:32:53] [STEP] ArgoCD Access Information ============================================================================= [2026-04-21 17:32:54] [INFO] ArgoCD URL: https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com [2026-04-21 17:32:54] [INFO] Authentication: Use 'Login with OpenShift' button (OpenShift credentials) [2026-04-21 17:32:54] [SUBSTEP] Verifying ArgoCD route is accessible [2026-04-21 17:33:14] [WAITING] Waiting for route to respond (5s/120s) [2026-04-21 17:33:19] [SUCCESS] ArgoCD is accessible at https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com [ARGOCD_DEPLOY_JSON] {"status":"success","url":"https://openshift-gitops-server-openshift-gitops.apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com","namespace":"openshift-gitops","auth_method":"openshift"} ============================================================================= [2026-04-21 17:33:19] [STEP] OpenShift GitOps Deployment Complete ============================================================================= ============================================================================= [2026-04-21 17:33:19] [STEP] Phase 2: Bootstrapping Host Cluster ============================================================================= [2026-04-21 17:33:19] [INFO] Bootstrapping host cluster components [2026-04-21 17:33:19] [INFO] No preview.env file found - using environment variables only [2026-04-21 17:33:19] [SUBSTEP] Configuring Quality Dashboard secrets Setting secrets for Quality Dashboard namespace/quality-dashboard created secret/quality-dashboard-secrets created [2026-04-21 17:33:22] [SUCCESS] Quality Dashboard secrets configured [2026-04-21 17:33:22] [SUBSTEP] Configuring CI Helper App secret Creating secret for CI Helper App namespace/ci-helper-app created secret/ci-helper-app-secrets created [2026-04-21 17:33:25] [SUCCESS] CI Helper App secret configured [2026-04-21 17:33:25] [SUCCESS] Host cluster bootstrap complete ============================================================================= [2026-04-21 17:33:25] [STEP] Phase 3: Bootstrapping Member Cluster ============================================================================= [2026-04-21 17:33:25] [INFO] Bootstrapping member cluster components [2026-04-21 17:33:25] [INFO] No preview.env file found - using environment variables only [2026-04-21 17:33:25] [SUBSTEP] Configuring Pipeline Service secrets Setting secrets for pipeline-service tekton-results namespace already exists, skipping creation tekton-logging namespace already exists, skipping creation namespace/product-kubearchive-logging created Creating DB secret secret/tekton-results-database created Creating S3 secret secret/tekton-results-s3 created Creating MinIO config secret/minio-storage-configuration created Creating S3 secret secret/tekton-results-s3 created Creating MinIO config MinIO config already exists, skipping creation Creating Postgres TLS certs secret/postgresql-tls created configmap/rds-root-crt created [2026-04-21 17:33:28] [SUCCESS] Pipeline Service secrets configured [2026-04-21 17:33:28] [SUBSTEP] Configuring GitHub secrets [2026-04-21 17:33:28] [INFO] GitHub token provided - creating secrets namespace/application-service created Creating a has secret from legacy token secret/has-github-token created [2026-04-21 17:33:29] [SUCCESS] GitHub secrets configured [2026-04-21 17:33:29] [SUBSTEP] Configuring Image Controller secrets [2026-04-21 17:33:29] [INFO] Quay organization: redhat-appstudio-qe Creating a secret with a token for Image Controller namespace/image-controller created secret/quaytoken created [2026-04-21 17:33:30] [SUCCESS] Image Controller secrets configured [2026-04-21 17:33:30] [SUCCESS] Member cluster bootstrap complete ============================================================================= [2026-04-21 17:33:30] [STEP] Phase 4: Bootstrapping Common Components ============================================================================= [2026-04-21 17:33:30] [INFO] Bootstrapping common cluster components [2026-04-21 17:33:30] [INFO] No preview.env file found at /tmp/tmp.dle8pz85Yc/tmp/infra-deployments/hack/../hack/preview.env - using environment variables only [2026-04-21 17:33:30] [SUBSTEP] Configuring Docker Hub pull secret Configuring the cluster with a pull secret for Docker Hub Saved credentials for docker.io into /tmp/tmp.ZHTIr2FiE5 secret/pull-secret data updated Saved credentials for docker.io into /tmp/tmp.ZHTIr2FiE5 secret/docker-io-pull created [2026-04-21 17:33:31] [SUCCESS] Docker Hub pull secret configured [2026-04-21 17:33:31] [SUBSTEP] Configuring DORA metrics exporter secrets Setting secrets for Dora metrics exporter namespace/dora-metrics created secret/exporters-secret created [2026-04-21 17:33:32] [SUCCESS] DORA metrics exporter secrets configured [2026-04-21 17:33:32] [SUCCESS] Common cluster bootstrap complete ============================================================================= [2026-04-21 17:33:32] [STEP] Phase 5: Mode-Specific Deployment (preview) ============================================================================= [2026-04-21 17:33:32] [INFO] Deploying preview configuration ============================================================================= [2026-04-21 17:33:32] [STEP] Starting Konflux Preview Environment Setup ============================================================================= [2026-04-21 17:33:32] [INFO] Script: /tmp/tmp.dle8pz85Yc/tmp/infra-deployments/hack/../hack/preview.sh [2026-04-21 17:33:32] [INFO] Options: OBO=false, GRAFANA=false, EAAS=false [2026-04-21 17:33:32] [INFO] Start time: 2026-04-21 17:33:32 UTC ============================================================================= [2026-04-21 17:33:32] [STEP] Cluster Context Information ============================================================================= [2026-04-21 17:33:32] [INFO] OpenShift Version: 4.18.9 [2026-04-21 17:33:32] [INFO] API Server: https://api.kx-4310c8a077.2y37.p3.openshiftapps.com:443 [2026-04-21 17:33:32] [INFO] Cluster ID: e4104402-de31-42b7-a92a-6451d1c705f3 [2026-04-21 17:33:32] [INFO] Cluster Domain: apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com [2026-04-21 17:33:32] [INFO] Total Nodes: 3 [2026-04-21 17:33:33] [INFO] - Master nodes: 0 [2026-04-21 17:33:33] [INFO] - Worker nodes: 3 [2026-04-21 17:33:33] [SUCCESS] Cluster operators: All healthy [CLUSTER_CONTEXT_JSON] {"ocp_version":"4.18.9","api_server":"https://api.kx-4310c8a077.2y37.p3.openshiftapps.com:443","cluster_id":"e4104402-de31-42b7-a92a-6451d1c705f3","cluster_domain":"apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com","total_nodes":3,"master_nodes":0,"worker_nodes":3,"degraded_operators":0} Switched to a new branch 'preview-main-swve' [2026-04-21 17:33:33] [SUCCESS] Git environment initialized [2026-04-21 17:33:33] [INFO] - Repository URL: https://github.com/redhat-appstudio-qe/infra-deployments.git [2026-04-21 17:33:33] [INFO] - Source branch: main [2026-04-21 17:33:33] [INFO] - Preview branch: preview-main-swve [2026-04-21 17:33:33] [INFO] - GitHub org: redhat-appstudio-qe ============================================================================= [2026-04-21 17:33:33] [STEP] Patching ArgoCD application manifests to use fork repository ============================================================================= [2026-04-21 17:33:33] [INFO] Setting repoURL to: https://github.com/redhat-appstudio-qe/infra-deployments.git [2026-04-21 17:33:33] [INFO] Setting targetRevision to: preview-main-swve [2026-04-21 17:33:33] [SUBSTEP] Patched: application-patch.yaml [2026-04-21 17:33:33] [SUBSTEP] Patched: application-set-patch.yaml [2026-04-21 17:33:33] [SUBSTEP] Patched: application-set-multisrc-src-1-patch.yaml [2026-04-21 17:33:33] [SUCCESS] All ArgoCD patch files updated ============================================================================= [2026-04-21 17:33:33] [STEP] Labeling cluster nodes for Konflux workloads ============================================================================= [2026-04-21 17:33:33] [INFO] Reference: https://github.com/redhat-appstudio/infra-deployments/pull/4415 [2026-04-21 17:33:33] [INFO] Found 3 nodes to label with 'konflux-ci.dev/workload=konflux-tenants' [2026-04-21 17:33:33] [SUBSTEP] Labeling node/ip-10-0-139-113.ec2.internal node/ip-10-0-139-113.ec2.internal labeled [2026-04-21 17:33:33] [SUCCESS] Successfully labeled node/ip-10-0-139-113.ec2.internal [2026-04-21 17:33:33] [SUBSTEP] Labeling node/ip-10-0-152-96.ec2.internal node/ip-10-0-152-96.ec2.internal labeled [2026-04-21 17:33:34] [SUCCESS] Successfully labeled node/ip-10-0-152-96.ec2.internal [2026-04-21 17:33:34] [SUBSTEP] Labeling node/ip-10-0-162-253.ec2.internal node/ip-10-0-162-253.ec2.internal labeled [2026-04-21 17:33:34] [SUCCESS] Successfully labeled node/ip-10-0-162-253.ec2.internal [2026-04-21 17:33:34] [SUBSTEP] Verifying labels on all nodes... [2026-04-21 17:33:34] [SUCCESS] All 3 nodes labeled and verified successfully ============================================================================= [2026-04-21 17:33:34] [STEP] Checking OCP version for Kueue compatibility ============================================================================= [2026-04-21 17:33:34] [INFO] Detected OpenShift Container Platform version: 4.18.9 (minor: 18) [2026-04-21 17:33:34] [SUCCESS] OCP version 4.18.9 meets Kueue requirements - Kueue will be deployed ============================================================================= [2026-04-21 17:33:34] [STEP] Configuring GitHub organization ============================================================================= [2026-04-21 17:33:34] [INFO] Setting GitHub org to: redhat-appstudio-qe [2026-04-21 17:33:34] [INFO] Configuring GitHub organization for Application Service (HAS) [2026-04-21 17:33:34] [INFO] - Target org: redhat-appstudio-qe [2026-04-21 17:33:34] [INFO] - Config file: /tmp/tmp.dle8pz85Yc/tmp/infra-deployments/hack/../components/has/base/kustomization.yaml [2026-04-21 17:33:34] [SUCCESS] GitHub organization set to 'redhat-appstudio-qe' [2026-04-21 17:33:34] [SUCCESS] Verified: Configuration updated correctly [2026-04-21 17:33:34] [SUCCESS] GitHub organization configured ============================================================================= [2026-04-21 17:33:34] [STEP] Configuring Rekor server hostname ============================================================================= [2026-04-21 17:33:34] [INFO] Cluster domain: apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com [2026-04-21 17:33:34] [INFO] Rekor server hostname: rekor.apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com [2026-04-21 17:33:34] [SUCCESS] Rekor server hostname configured ============================================================================= [2026-04-21 17:33:34] [STEP] Applying service image overrides from environment variables ============================================================================= [2026-04-21 17:33:34] [INFO] No service image overrides configured - using default images ============================================================================= [2026-04-21 17:33:34] [STEP] Committing and pushing preview changes ============================================================================= [preview-main-swve b5b39c13e] Preview mode, do not merge into main 8 files changed, 15 insertions(+), 21 deletions(-) remote: remote: Create a pull request for 'preview-main-swve' on GitHub by visiting: remote: https://github.com/redhat-appstudio-qe/infra-deployments/pull/new/preview-main-swve remote: To https://github.com/redhat-appstudio-qe/infra-deployments.git * [new branch] preview-main-swve -> preview-main-swve branch 'preview-main-swve' set up to track 'qe/preview-main-swve'. [2026-04-21 17:33:35] [SUCCESS] Preview changes committed and pushed to qe/preview-main-swve ============================================================================= [2026-04-21 17:33:35] [STEP] Deploying ArgoCD applications ============================================================================= [2026-04-21 17:33:35] [SUBSTEP] Applying root Application from: /tmp/tmp.dle8pz85Yc/tmp/infra-deployments/hack/../argo-cd-apps/app-of-app-sets/development application.argoproj.io/all-application-sets created [2026-04-21 17:33:36] [SUCCESS] Root Application 'all-application-sets' created [2026-04-21 17:33:36] [SUBSTEP] Waiting for 'all-application-sets' to become Healthy and Synced [2026-04-21 17:33:36] [WAITING] Root application status: ' ' (target: 'Healthy Synced') - 5s elapsed [2026-04-21 17:33:41] [WAITING] Root application status: ' ' (target: 'Healthy Synced') - 10s elapsed [2026-04-21 17:33:46] [WAITING] Root application status: 'Healthy OutOfSync' (target: 'Healthy Synced') - 15s elapsed [2026-04-21 17:33:51] [WAITING] Root application status: 'Healthy OutOfSync' (target: 'Healthy Synced') - 20s elapsed [2026-04-21 17:33:57] [SUCCESS] Root application 'all-application-sets' is Healthy and Synced [2026-04-21 17:33:57] [SUBSTEP] Triggering hard refresh on all ArgoCD applications [2026-04-21 17:33:57] [INFO] Found 43 applications to refresh application.argoproj.io/disable-csvcopy-in-cluster-local patched application.argoproj.io/has-in-cluster-local patched application.argoproj.io/build-service-in-cluster-local patched application.argoproj.io/image-rbac-proxy-in-cluster-local patched application.argoproj.io/crossplane-control-plane-in-cluster-local patched application.argoproj.io/dora-metrics-in-cluster-local patched application.argoproj.io/monitoring-cardinality-in-cluster-local patched application.argoproj.io/kueue-in-cluster-local patched application.argoproj.io/disaster-recovery-in-cluster-local patched application.argoproj.io/all-application-sets patched application.argoproj.io/squid-in-cluster-local patched application.argoproj.io/smee-client-in-cluster-local patched application.argoproj.io/mintmaker-in-cluster-local patched application.argoproj.io/policies-in-cluster-local patched application.argoproj.io/tempo-in-cluster-local patched application.argoproj.io/tracing-workload-tracing-in-cluster-local patched application.argoproj.io/vector-tekton-logs-collector-in-cluster-local patched application.argoproj.io/pipeline-service-in-cluster-local patched application.argoproj.io/kubearchive-in-cluster-local patched application.argoproj.io/repository-validator-in-cluster-local patched application.argoproj.io/application-api-in-cluster-local patched application.argoproj.io/vector-kubearchive-log-collector-in-cluster-local patched (no change) application.argoproj.io/internal-services-in-cluster-local patched application.argoproj.io/konflux-info-in-cluster-local patched application.argoproj.io/enterprise-contract-in-cluster-local patched application.argoproj.io/build-templates-in-cluster-local patched application.argoproj.io/image-controller-in-cluster-local patched application.argoproj.io/knative-eventing-in-cluster-local patched application.argoproj.io/perf-team-prometheus-reader-in-cluster-local patched application.argoproj.io/release-in-cluster-local patched application.argoproj.io/backup-in-cluster-local patched application.argoproj.io/tracing-workload-otel-collector-in-cluster-local patched application.argoproj.io/konflux-rbac-in-cluster-local patched application.argoproj.io/konflux-kite-in-cluster-local patched application.argoproj.io/monitoring-workload-custom-kube-state-metrics-in-cluster-local patched application.argoproj.io/trust-manager-in-cluster-local patched application.argoproj.io/integration-in-cluster-local patched application.argoproj.io/project-controller-in-cluster-local patched application.argoproj.io/monitoring-workload-prometheus-in-cluster-local patched application.argoproj.io/cert-manager-in-cluster-local patched application.argoproj.io/monitoring-registry-in-cluster-local patched application.argoproj.io/kyverno-in-cluster-local patched application.argoproj.io/multi-platform-controller-in-cluster-local patched [2026-04-21 17:34:04] [SUCCESS] Hard refresh triggered on all 43 applications [2026-04-21 17:34:04] [SUBSTEP] Waiting for refresh operations to complete [2026-04-21 17:34:04] [PROGRESS] Refresh: 0/43 complete | 43 still refreshing (5s elapsed) [2026-04-21 17:34:09] [PROGRESS] Refresh: 0/43 complete | 43 still refreshing (10s elapsed) [2026-04-21 17:34:14] [PROGRESS] Refresh: 0/43 complete | 43 still refreshing (15s elapsed) [2026-04-21 17:34:20] [PROGRESS] Refresh: 11/43 complete | 32 still refreshing (20s elapsed) [2026-04-21 17:34:25] [PROGRESS] Refresh: 30/43 complete | 13 still refreshing (25s elapsed) [2026-04-21 17:34:31] [PROGRESS] Refresh: 30/43 complete | 13 still refreshing (30s elapsed) [2026-04-21 17:34:36] [PROGRESS] Refresh: 35/43 complete | 8 still refreshing (35s elapsed) [2026-04-21 17:34:41] [PROGRESS] Refresh: 37/43 complete | 6 still refreshing (40s elapsed) [2026-04-21 17:34:47] [PROGRESS] Refresh: 39/43 complete | 4 still refreshing (45s elapsed) [2026-04-21 17:34:52] [PROGRESS] Refresh: 40/43 complete | 3 still refreshing (50s elapsed) [2026-04-21 17:34:58] [PROGRESS] Refresh: 40/43 complete | 3 still refreshing (55s elapsed) [2026-04-21 17:35:03] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (60s elapsed) [2026-04-21 17:35:08] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (65s elapsed) [2026-04-21 17:35:14] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (70s elapsed) [2026-04-21 17:35:19] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (75s elapsed) [2026-04-21 17:35:24] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (80s elapsed) [2026-04-21 17:35:29] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (85s elapsed) [2026-04-21 17:35:35] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (90s elapsed) [2026-04-21 17:35:40] [PROGRESS] Refresh: 42/43 complete | 1 still refreshing (95s elapsed) [2026-04-21 17:35:45] [SUCCESS] All 43 applications refreshed ============================================================================= [2026-04-21 17:35:45] [STEP] Waiting for all ArgoCD applications to sync and become healthy ============================================================================= [2026-04-21 17:35:45] [INFO] Timeout: 2700 seconds (45 minutes) [2026-04-21 17:35:46] [PROGRESS] Applications: 12/43 ready | 31 pending (0m 0s elapsed) [2026-04-21 17:35:46] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,has-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-otel-collector-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:35:46] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:35:56] [PROGRESS] Applications: 12/43 ready | 31 pending (0m 11s elapsed) [2026-04-21 17:35:56] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,has-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-otel-collector-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:35:56] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:36:06] [PROGRESS] Applications: 11/44 ready | 33 pending (0m 21s elapsed) [2026-04-21 17:36:06] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,cert-manager-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,has-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,postgres,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-otel-collector-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:36:06] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:36:16] [PROGRESS] Applications: 12/44 ready | 32 pending (0m 31s elapsed) [2026-04-21 17:36:16] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,has-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,postgres,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-otel-collector-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:36:16] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:36:26] [PROGRESS] Applications: 12/44 ready | 32 pending (0m 41s elapsed) [2026-04-21 17:36:26] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,cert-manager-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,postgres,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-otel-collector-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:36:26] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:36:36] [PROGRESS] Applications: 13/44 ready | 31 pending (0m 51s elapsed) [2026-04-21 17:36:36] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,postgres,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-otel-collector-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:36:36] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:36:47] [PROGRESS] Applications: 13/44 ready | 31 pending (1m 1s elapsed) [2026-04-21 17:36:47] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,postgres,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-otel-collector-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:36:47] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:36:57] [PROGRESS] Applications: 14/44 ready | 30 pending (1m 12s elapsed) [2026-04-21 17:36:57] [INFO] Pending: backup-in-cluster-local,build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,image-controller-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,postgres,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:36:57] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:37:07] [PROGRESS] Applications: 18/44 ready | 26 pending (1m 22s elapsed) [2026-04-21 17:37:07] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,smee-client-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:37:07] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:37:17] [PROGRESS] Applications: 18/44 ready | 26 pending (1m 32s elapsed) [2026-04-21 17:37:17] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:37:17] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:37:27] [PROGRESS] Applications: 18/44 ready | 26 pending (1m 42s elapsed) [2026-04-21 17:37:27] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:37:27] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:37:38] [PROGRESS] Applications: 18/44 ready | 26 pending (1m 52s elapsed) [2026-04-21 17:37:38] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,image-rbac-proxy-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:37:38] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:37:48] [PROGRESS] Applications: 19/44 ready | 25 pending (2m 3s elapsed) [2026-04-21 17:37:48] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-info-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:37:48] [SUBSTEP] Detailed status of pending applications: [2026-04-21 17:37:48] [INFO] ├─ App: build-service-in-cluster-local [2026-04-21 17:37:48] [INFO] │ ├─ Sync Status: Synced [2026-04-21 17:37:48] [INFO] │ ├─ Health Status: Progressing [2026-04-21 17:37:48] [INFO] │ └─ Message: No message [2026-04-21 17:37:48] [INFO] ├─ App: crossplane-control-plane-in-cluster-local [2026-04-21 17:37:48] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:48] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:48] [INFO] │ ├─ Out-of-sync resources: 3 [2026-04-21 17:37:48] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:48] [WARN] │ │ └─ Secret/provider-kubernetes-in-cluster: Missing [2026-04-21 17:37:48] [WARN] │ │ └─ ClusterRole/namespace-claim-cleaner: Missing [2026-04-21 17:37:48] [WARN] │ │ └─ ClusterRole/provider-kubernetes-extra: Missing [2026-04-21 17:37:48] [INFO] │ └─ Message: No message [2026-04-21 17:37:48] [INFO] ├─ App: disaster-recovery-in-cluster-local [2026-04-21 17:37:48] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:48] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:48] [INFO] │ ├─ Out-of-sync resources: 11 [2026-04-21 17:37:48] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:48] [WARN] │ │ └─ Namespace/konflux-disaster-recovery: Missing [2026-04-21 17:37:48] [WARN] │ │ └─ ServiceAccount/cron-trigger: Missing [2026-04-21 17:37:48] [WARN] │ │ └─ CronJob/run-disaster-recovery-pipelinerun: Missing [2026-04-21 17:37:48] [INFO] │ └─ Message: No message [2026-04-21 17:37:49] [INFO] ├─ App: integration-in-cluster-local [2026-04-21 17:37:49] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:49] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:49] [INFO] │ ├─ Out-of-sync resources: 19 [2026-04-21 17:37:49] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:49] [WARN] │ │ └─ CustomResourceDefinition/componentgroups.appstudio.redhat.com: Missing [2026-04-21 17:37:49] [WARN] │ │ └─ CustomResourceDefinition/integrationtestscenarios.appstudio.redhat.com: Missing [2026-04-21 17:37:49] [WARN] │ │ └─ ClusterRole/crd-manager-for-integration: Missing [2026-04-21 17:37:49] [INFO] │ └─ Message: No message [2026-04-21 17:37:49] [INFO] ├─ App: knative-eventing-in-cluster-local [2026-04-21 17:37:49] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:49] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:49] [INFO] │ ├─ Out-of-sync resources: 54 [2026-04-21 17:37:49] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:49] [WARN] │ │ └─ CustomResourceDefinition/apiserversources.sources.knative.dev: Missing [2026-04-21 17:37:49] [WARN] │ │ └─ CustomResourceDefinition/brokers.eventing.knative.dev: Missing [2026-04-21 17:37:49] [WARN] │ │ └─ CustomResourceDefinition/channels.messaging.knative.dev: Missing [2026-04-21 17:37:49] [INFO] │ └─ Message: No message [2026-04-21 17:37:49] [INFO] ├─ App: konflux-info-in-cluster-local [2026-04-21 17:37:49] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:49] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:49] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:37:49] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:49] [WARN] │ │ └─ RoleBinding/konflux-public-info-view-rb: Missing [2026-04-21 17:37:49] [INFO] │ └─ Message: No message [2026-04-21 17:37:49] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-04-21 17:37:49] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:49] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:49] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:37:49] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:49] [WARN] │ │ └─ ClusterRole/konflux-kite-authorizer: Missing [2026-04-21 17:37:49] [INFO] │ └─ Message: No message [2026-04-21 17:37:50] [INFO] ├─ App: konflux-rbac-in-cluster-local [2026-04-21 17:37:50] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:50] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:50] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:37:50] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:50] [WARN] │ │ └─ ClusterRole/appstudio-pipelines-runner: Missing [2026-04-21 17:37:50] [WARN] │ │ └─ ClusterRole/konflux-integration-runner: Missing [2026-04-21 17:37:50] [INFO] │ └─ Message: No message [2026-04-21 17:37:50] [INFO] ├─ App: kubearchive-in-cluster-local [2026-04-21 17:37:50] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:50] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:50] [INFO] │ ├─ Out-of-sync resources: 23 [2026-04-21 17:37:50] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:50] [WARN] │ │ └─ Secret/kubearchive-logging: Missing [2026-04-21 17:37:50] [WARN] │ │ └─ Secret/metrics-reader: Missing [2026-04-21 17:37:50] [WARN] │ │ └─ CustomResourceDefinition/clusterkubearchiveconfigs.kubearchive.org: Missing [2026-04-21 17:37:50] [INFO] │ └─ Message: No message [2026-04-21 17:37:50] [INFO] ├─ App: kueue-in-cluster-local [2026-04-21 17:37:50] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:50] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:50] [INFO] │ ├─ Out-of-sync resources: 54 [2026-04-21 17:37:50] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:50] [WARN] │ │ └─ ConfigMap/alert-mgr-kueue-admission-config: Missing [2026-04-21 17:37:50] [WARN] │ │ └─ ConfigMap/tekton-kueue-config: Missing [2026-04-21 17:37:50] [WARN] │ │ └─ Namespace/kueue-external-admission: Missing [2026-04-21 17:37:50] [INFO] │ └─ Message: No message [2026-04-21 17:37:50] [INFO] ├─ App: kyverno-in-cluster-local [2026-04-21 17:37:50] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:50] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:50] [INFO] │ ├─ Out-of-sync resources: 40 [2026-04-21 17:37:50] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:50] [WARN] │ │ └─ ServiceAccount/kyverno-admission-controller: Missing [2026-04-21 17:37:50] [WARN] │ │ └─ ServiceAccount/kyverno-background-controller: Missing [2026-04-21 17:37:50] [WARN] │ │ └─ ServiceAccount/kyverno-cleanup-controller: Missing [2026-04-21 17:37:50] [INFO] │ └─ Message: No message [2026-04-21 17:37:51] [INFO] ├─ App: mintmaker-in-cluster-local [2026-04-21 17:37:51] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:51] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:51] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:37:51] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:51] [WARN] │ │ └─ CustomResourceDefinition/dependencyupdatechecks.appstudio.redhat.com: Missing [2026-04-21 17:37:51] [INFO] │ └─ Message: No message [2026-04-21 17:37:51] [INFO] ├─ App: monitoring-registry-in-cluster-local [2026-04-21 17:37:51] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:51] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:51] [INFO] │ ├─ Out-of-sync resources: 3 [2026-04-21 17:37:51] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:51] [WARN] │ │ └─ ClusterRole/kube-rbac-proxy-registry: Missing [2026-04-21 17:37:51] [WARN] │ │ └─ ClusterRole/registry-exporter-role-metrics-reader: Missing [2026-04-21 17:37:51] [WARN] │ │ └─ RoleBinding/appstudio-registry-exporter-maintainers: Missing [2026-04-21 17:37:51] [INFO] │ └─ Message: No message [2026-04-21 17:37:51] [INFO] ├─ App: monitoring-workload-prometheus-in-cluster-local [2026-04-21 17:37:51] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:51] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:51] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:37:51] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:51] [WARN] │ │ └─ ClusterRole/kube-rbac-proxy: Missing [2026-04-21 17:37:51] [WARN] │ │ └─ ClusterRole/o11y-dummy-service-metrics-reader: Missing [2026-04-21 17:37:51] [INFO] │ └─ Message: No message [2026-04-21 17:37:51] [INFO] ├─ App: multi-platform-controller-in-cluster-local [2026-04-21 17:37:51] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:51] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:51] [INFO] │ ├─ Out-of-sync resources: 19 [2026-04-21 17:37:51] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:51] [WARN] │ │ └─ ConfigMap/clean-shared-host-script-4t4gt46c47: Missing [2026-04-21 17:37:51] [WARN] │ │ └─ ConfigMap/host-config: Missing [2026-04-21 17:37:51] [WARN] │ │ └─ ConfigMap/provision-macos-host-script-hm2gmgm9b2: Missing [2026-04-21 17:37:51] [INFO] │ └─ Message: No message [2026-04-21 17:37:52] [INFO] ├─ App: perf-team-prometheus-reader-in-cluster-local [2026-04-21 17:37:52] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:52] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:52] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:37:52] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:52] [WARN] │ │ └─ ClusterRole/perf-team-prometheus-reader-oomcrash-role: Missing [2026-04-21 17:37:52] [INFO] │ └─ Message: No message [2026-04-21 17:37:52] [INFO] ├─ App: pipeline-service-in-cluster-local [2026-04-21 17:37:52] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:52] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:52] [INFO] │ ├─ Out-of-sync resources: 34 [2026-04-21 17:37:52] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:52] [WARN] │ │ └─ Secret/console-sa-secret: Missing [2026-04-21 17:37:52] [WARN] │ │ └─ Secret/repo-bitnami-postgresql: Missing [2026-04-21 17:37:52] [WARN] │ │ └─ Secret/metrics-reader: Missing [2026-04-21 17:37:52] [INFO] │ └─ Message: No message [2026-04-21 17:37:52] [INFO] ├─ App: policies-in-cluster-local [2026-04-21 17:37:52] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:52] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:52] [INFO] │ ├─ Out-of-sync resources: 33 [2026-04-21 17:37:52] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:52] [WARN] │ │ └─ ValidatingAdmissionPolicy/no-skip.kueue.konflux-ci.dev: Missing [2026-04-21 17:37:52] [WARN] │ │ └─ ValidatingAdmissionPolicyBinding/no-skip.kueue.konflux-ci.dev: Missing [2026-04-21 17:37:52] [WARN] │ │ └─ ClusterPolicy/bootstrap-tenant-namespace-queue: Missing [2026-04-21 17:37:52] [INFO] │ └─ Message: No message [2026-04-21 17:37:52] [INFO] ├─ App: project-controller-in-cluster-local [2026-04-21 17:37:52] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:52] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:52] [INFO] │ ├─ Out-of-sync resources: 7 [2026-04-21 17:37:52] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:52] [WARN] │ │ └─ CustomResourceDefinition/projectdevelopmentstreams.projctl.konflux.dev: Missing [2026-04-21 17:37:52] [WARN] │ │ └─ CustomResourceDefinition/projectdevelopmentstreamtemplates.projctl.konflux.dev: Missing [2026-04-21 17:37:52] [WARN] │ │ └─ CustomResourceDefinition/projects.projctl.konflux.dev: Missing [2026-04-21 17:37:52] [INFO] │ └─ Message: No message [2026-04-21 17:37:53] [INFO] ├─ App: release-in-cluster-local [2026-04-21 17:37:53] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:53] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:53] [INFO] │ ├─ Out-of-sync resources: 26 [2026-04-21 17:37:53] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:53] [WARN] │ │ └─ CustomResourceDefinition/releaseplanadmissions.appstudio.redhat.com: Missing [2026-04-21 17:37:53] [WARN] │ │ └─ CustomResourceDefinition/releaseplans.appstudio.redhat.com: Missing [2026-04-21 17:37:53] [WARN] │ │ └─ CustomResourceDefinition/releases.appstudio.redhat.com: Missing [2026-04-21 17:37:53] [INFO] │ └─ Message: No message [2026-04-21 17:37:53] [INFO] ├─ App: squid-in-cluster-local [2026-04-21 17:37:53] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:53] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:53] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:37:53] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:53] [WARN] │ │ └─ ServiceAccount/squid: Missing [2026-04-21 17:37:53] [WARN] │ │ └─ RoleBinding/konflux-vanguard-admins: Missing [2026-04-21 17:37:53] [INFO] │ └─ Message: No message [2026-04-21 17:37:53] [INFO] ├─ App: tempo-in-cluster-local [2026-04-21 17:37:53] [INFO] │ ├─ Sync Status: Synced [2026-04-21 17:37:53] [INFO] │ ├─ Health Status: Degraded [2026-04-21 17:37:53] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:53] [WARN] │ │ └─ Subscription/tempo-product: Degraded [2026-04-21 17:37:53] [INFO] │ └─ Message: No message [2026-04-21 17:37:53] [INFO] ├─ App: trust-manager-in-cluster-local [2026-04-21 17:37:53] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:53] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:53] [INFO] │ ├─ Out-of-sync resources: 5 [2026-04-21 17:37:53] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:53] [WARN] │ │ └─ ServiceAccount/trust-manager: Missing [2026-04-21 17:37:53] [WARN] │ │ └─ CustomResourceDefinition/bundles.trust.cert-manager.io: Missing [2026-04-21 17:37:53] [WARN] │ │ └─ ClusterRole/trust-manager: Missing [2026-04-21 17:37:53] [INFO] │ └─ Message: No message [2026-04-21 17:37:53] [INFO] ├─ App: vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:37:53] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:53] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:37:53] [INFO] │ ├─ Out-of-sync resources: 53 [2026-04-21 17:37:53] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:53] [WARN] │ │ └─ ConfigMap/grafana: Missing [2026-04-21 17:37:53] [WARN] │ │ └─ ConfigMap/loki: Missing [2026-04-21 17:37:53] [WARN] │ │ └─ ConfigMap/loki-gateway: Missing [2026-04-21 17:37:53] [WARN] │ └─ Conditions: [2026-04-21 17:37:53] [WARN] │ └─ [RepeatedResourceWarning] Resource /ServiceAccount/product-kubearchive-logging/minio-sa appeared 2 times among application resources. [2026-04-21 17:37:54] [INFO] ├─ App: vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:37:54] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:37:54] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:37:54] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:37:54] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:37:54] [WARN] │ │ └─ ClusterRole/vector-tekton-logs-collector: Missing [2026-04-21 17:37:54] [INFO] │ └─ Message: No message [2026-04-21 17:37:54] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:38:04] [PROGRESS] Applications: 20/44 ready | 24 pending (2m 19s elapsed) [2026-04-21 17:38:04] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:38:04] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:38:14] [PROGRESS] Applications: 20/44 ready | 24 pending (2m 29s elapsed) [2026-04-21 17:38:14] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:38:14] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:38:24] [PROGRESS] Applications: 20/44 ready | 24 pending (2m 39s elapsed) [2026-04-21 17:38:24] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:38:24] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:38:34] [PROGRESS] Applications: 20/44 ready | 24 pending (2m 49s elapsed) [2026-04-21 17:38:34] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,mintmaker-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:38:34] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:38:45] [PROGRESS] Applications: 21/44 ready | 23 pending (2m 59s elapsed) [2026-04-21 17:38:45] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:38:45] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:38:55] [PROGRESS] Applications: 20/44 ready | 24 pending (3m 10s elapsed) [2026-04-21 17:38:55] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,tracing-workload-tracing-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:38:55] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:39:05] [PROGRESS] Applications: 21/44 ready | 23 pending (3m 20s elapsed) [2026-04-21 17:39:05] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,squid-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:39:05] [WARN] Found applications in Unknown state (not Progressing), investigating... [2026-04-21 17:39:05] [ERROR] Application 'vector-kubearchive-log-collector-in-cluster-local' is in Unknown state without 'context deadline exceeded' [2026-04-21 17:39:05] [INFO] ├─ App: vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:39:05] [INFO] │ ├─ Sync Status: Unknown [2026-04-21 17:39:05] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:05] [INFO] │ ├─ Out-of-sync resources: 13 [2026-04-21 17:39:05] [WARN] │ └─ Conditions: [2026-04-21 17:39:06] [WARN] │ └─ [ComparisonError] Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = `kustomize build /components/vector-kubearchive-log-collector/development --enable-helm --helm-kube-version 1.31 --helm-api-versions admissionregistration.k8s.io/v1 --helm-api-versions admissionregistration.k8s.io/v1/MutatingWebhookConfiguration --helm-api-versions admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy --helm-api-versions admissionregistration.k8s.io/v1/ValidatingAdmissionPolicyBinding --helm-api-versions admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration --helm-api-versions admissionregistration.k8s.io/v1beta1 --helm-api-versions admissionregistration.k8s.io/v1beta1/ValidatingAdmissionPolicy --helm-api-versions admissionregistration.k8s.io/v1beta1/ValidatingAdmissionPolicyBinding --helm-api-versions apiextensions.crossplane.io/v1 --helm-api-versions apiextensions.crossplane.io/v1/CompositeResourceDefinition --helm-api-versions apiextensions.crossplane.io/v2 --helm-api-versions apiextensions.crossplane.io/v2/CompositeResourceDefinition --helm-api-versions apiextensions.k8s.io/v1 --helm-api-versions apiextensions.k8s.io/v1/CustomResourceDefinition --helm-api-versions apiregistration.k8s.io/v1 --helm-api-versions apiregistration.k8s.io/v1/APIService --helm-api-versions apiserver.openshift.io/v1 --helm-api-versions apiserver.openshift.io/v1/APIRequestCount --helm-api-versions apps.openshift.io/v1 --helm-api-versions apps.openshift.io/v1/DeploymentConfig --helm-api-versions apps/v1 --helm-api-versions apps/v1/ControllerRevision --helm-api-versions apps/v1/DaemonSet --helm-api-versions apps/v1/Deployment --helm-api-versions apps/v1/ReplicaSet --helm-api-versions apps/v1/StatefulSet --helm-api-versions appstudio.redhat.com/v1alpha1 --helm-api-versions appstudio.redhat.com/v1alpha1/Application --helm-api-versions appstudio.redhat.com/v1alpha1/Component --helm-api-versions appstudio.redhat.com/v1alpha1/ComponentDetectionQuery --helm-api-versions appstudio.redhat.com/v1alpha1/DependencyUpdateCheck --helm-api-versions appstudio.redhat.com/v1alpha1/DeploymentTarget --helm-api-versions appstudio.redhat.com/v1alpha1/DeploymentTargetClaim --helm-api-versions appstudio.redhat.com/v1alpha1/DeploymentTargetClass --helm-api-versions appstudio.redhat.com/v1alpha1/EnterpriseContractPolicy --helm-api-versions appstudio.redhat.com/v1alpha1/Environment --helm-api-versions appstudio.redhat.com/v1alpha1/ImageRepository --helm-api-versions appstudio.redhat.com/v1alpha1/InternalRequest --helm-api-versions appstudio.redhat.com/v1alpha1/InternalServicesConfig --helm-api-versions appstudio.redhat.com/v1alpha1/PromotionRun --helm-api-versions appstudio.redhat.com/v1alpha1/Snapshot --helm-api-versions appstudio.redhat.com/v1alpha1/SnapshotEnvironmentBinding --helm-api-versions argoproj.io/v1alpha1 --helm-api-versions argoproj.io/v1alpha1/AnalysisRun --helm-api-versions argoproj.io/v1alpha1/AnalysisTemplate --helm-api-versions argoproj.io/v1alpha1/AppProject --helm-api-versions argoproj.io/v1alpha1/Application --helm-api-versions argoproj.io/v1alpha1/ApplicationSet --helm-api-versions argoproj.io/v1alpha1/ArgoCD --helm-api-versions argoproj.io/v1alpha1/ClusterAnalysisTemplate --helm-api-versions argoproj.io/v1alpha1/Experiment --helm-api-versions argoproj.io/v1alpha1/NotificationsConfiguration --helm-api-versions argoproj.io/v1alpha1/Rollout --helm-api-versions argoproj.io/v1alpha1/RolloutManager --helm-api-versions argoproj.io/v1beta1 --helm-api-versions argoproj.io/v1beta1/ArgoCD --helm-api-versions authorization.openshift.io/v1 --helm-api-versions authorization.openshift.io/v1/RoleBindingRestriction --helm-api-versions autoscaling/v1 --helm-api-versions autoscaling/v1/HorizontalPodAutoscaler --helm-api-versions autoscaling/v2 --helm-api-versions autoscaling/v2/HorizontalPodAutoscaler --helm-api-versions batch/v1 --helm-api-versions batch/v1/CronJob --helm-api-versions batch/v1/Job --helm-api-versions build.openshift.io/v1 --helm-api-versions build.openshift.io/v1/Build --helm-api-versions build.openshift.io/v1/BuildConfig --helm-api-versions certificates.k8s.io/v1 --helm-api-versions certificates.k8s.io/v1/CertificateSigningRequest --helm-api-versions cloud.network.openshift.io/v1 --helm-api-versions cloud.network.openshift.io/v1/CloudPrivateIPConfig --helm-api-versions cloudcredential.openshift.io/v1 --helm-api-versions cloudcredential.openshift.io/v1/CredentialsRequest --helm-api-versions cluster.open-cluster-management.io/v1alpha1 --helm-api-versions cluster.open-cluster-management.io/v1alpha1/ClusterClaim --helm-api-versions config.openshift.io/v1 --helm-api-versions config.openshift.io/v1/APIServer --helm-api-versions config.openshift.io/v1/Authentication --helm-api-versions config.openshift.io/v1/Build --helm-api-versions config.openshift.io/v1/ClusterOperator --helm-api-versions config.openshift.io/v1/ClusterVersion --helm-api-versions config.openshift.io/v1/Console --helm-api-versions config.openshift.io/v1/DNS --helm-api-versions config.openshift.io/v1/FeatureGate --helm-api-versions config.openshift.io/v1/Image --helm-api-versions config.openshift.io/v1/ImageContentPolicy --helm-api-versions config.openshift.io/v1/ImageDigestMirrorSet --helm-api-versions config.openshift.io/v1/ImageTagMirrorSet --helm-api-versions config.openshift.io/v1/Infrastructure --helm-api-versions config.openshift.io/v1/Ingress --helm-api-versions config.openshift.io/v1/Network --helm-api-versions config.openshift.io/v1/Node --helm-api-versions config.openshift.io/v1/OAuth --helm-api-versions config.openshift.io/v1/OperatorHub --helm-api-versions config.openshift.io/v1/Project --helm-api-versions config.openshift.io/v1/Proxy --helm-api-versions config.openshift.io/v1/Scheduler --helm-api-versions console.openshift.io/v1 --helm-api-versions console.openshift.io/v1/ConsoleCLIDownload --helm-api-versions console.openshift.io/v1/ConsoleExternalLogLink --helm-api-versions console.openshift.io/v1/ConsoleLink --helm-api-versions console.openshift.io/v1/ConsoleNotification --helm-api-versions console.openshift.io/v1/ConsolePlugin --helm-api-versions console.openshift.io/v1/ConsoleQuickStart --helm-api-versions console.openshift.io/v1/ConsoleSample --helm-api-versions console.openshift.io/v1/ConsoleYAMLSample --helm-api-versions console.openshift.io/v1alpha1 --helm-api-versions console.openshift.io/v1alpha1/ConsolePlugin --helm-api-versions controlplane.operator.openshift.io/v1alpha1 --helm-api-versions controlplane.operator.openshift.io/v1alpha1/PodNetworkConnectivityCheck --helm-api-versions coordination.k8s.io/v1 --helm-api-versions coordination.k8s.io/v1/Lease --helm-api-versions discovery.k8s.io/v1 --helm-api-versions discovery.k8s.io/v1/EndpointSlice --helm-api-versions events.k8s.io/v1 --helm-api-versions events.k8s.io/v1/Event --helm-api-versions flowcontrol.apiserver.k8s.io/v1 --helm-api-versions flowcontrol.apiserver.k8s.io/v1/FlowSchema --helm-api-versions flowcontrol.apiserver.k8s.io/v1/PriorityLevelConfiguration --helm-api-versions flowcontrol.apiserver.k8s.io/v1beta3 --helm-api-versions flowcontrol.apiserver.k8s.io/v1beta3/FlowSchema --helm-api-versions flowcontrol.apiserver.k8s.io/v1beta3/PriorityLevelConfiguration --helm-api-versions helm.openshift.io/v1beta1 --helm-api-versions helm.openshift.io/v1beta1/HelmChartRepository --helm-api-versions helm.openshift.io/v1beta1/ProjectHelmChartRepository --helm-api-versions image.openshift.io/v1 --helm-api-versions image.openshift.io/v1/Image --helm-api-versions image.openshift.io/v1/ImageStream --helm-api-versions imageregistry.operator.openshift.io/v1 --helm-api-versions imageregistry.operator.openshift.io/v1/Config --helm-api-versions imageregistry.operator.openshift.io/v1/ImagePruner --helm-api-versions ingress.operator.openshift.io/v1 --helm-api-versions ingress.operator.openshift.io/v1/DNSRecord --helm-api-versions k8s.cni.cncf.io/v1 --helm-api-versions k8s.cni.cncf.io/v1/NetworkAttachmentDefinition --helm-api-versions k8s.cni.cncf.io/v1alpha1 --helm-api-versions k8s.cni.cncf.io/v1alpha1/IPAMClaim --helm-api-versions k8s.ovn.org/v1 --helm-api-versions k8s.ovn.org/v1/AdminPolicyBasedExternalRoute --helm-api-versions k8s.ovn.org/v1/ClusterUserDefinedNetwork --helm-api-versions k8s.ovn.org/v1/EgressFirewall --helm-api-versions k8s.ovn.org/v1/EgressIP --helm-api-versions k8s.ovn.org/v1/EgressQoS --helm-api-versions k8s.ovn.org/v1/EgressService --helm-api-versions k8s.ovn.org/v1/UserDefinedNetwork --helm-api-versions machineconfiguration.openshift.io/v1 --helm-api-versions machineconfiguration.openshift.io/v1/ControllerConfig --helm-api-versions migration.k8s.io/v1alpha1 --helm-api-versions migration.k8s.io/v1alpha1/StorageState --helm-api-versions migration.k8s.io/v1alpha1/StorageVersionMigration --helm-api-versions monitoring.coreos.com/v1 --helm-api-versions monitoring.coreos.com/v1/Alertmanager --helm-api-versions monitoring.coreos.com/v1/PodMonitor --helm-api-versions monitoring.coreos.com/v1/Probe --helm-api-versions monitoring.coreos.com/v1/Prometheus --helm-api-versions monitoring.coreos.com/v1/PrometheusRule --helm-api-versions monitoring.coreos.com/v1/ServiceMonitor --helm-api-versions monitoring.coreos.com/v1/ThanosRuler --helm-api-versions monitoring.coreos.com/v1alpha1 --helm-api-versions monitoring.coreos.com/v1alpha1/AlertmanagerConfig --helm-api-versions monitoring.coreos.com/v1beta1 --helm-api-versions monitoring.coreos.com/v1beta1/AlertmanagerConfig --helm-api-versions monitoring.openshift.io/v1 --helm-api-versions monitoring.openshift.io/v1/AlertRelabelConfig --helm-api-versions monitoring.openshift.io/v1/AlertingRule --helm-api-versions network.operator.openshift.io/v1 --helm-api-versions network.operator.openshift.io/v1/EgressRouter --helm-api-versions network.operator.openshift.io/v1/OperatorPKI --helm-api-versions networking.k8s.io/v1 --helm-api-versions networking.k8s.io/v1/Ingress --helm-api-versions networking.k8s.io/v1/IngressClass --helm-api-versions networking.k8s.io/v1/NetworkPolicy --helm-api-versions node.k8s.io/v1 --helm-api-versions node.k8s.io/v1/RuntimeClass --helm-api-versions oadp.openshift.io/v1alpha1 --helm-api-versions oadp.openshift.io/v1alpha1/CloudStorage --helm-api-versions oadp.openshift.io/v1alpha1/DataProtectionApplication --helm-api-versions oauth.openshift.io/v1 --helm-api-versions oauth.openshift.io/v1/OAuthAccessToken --helm-api-versions oauth.openshift.io/v1/OAuthAuthorizeToken --helm-api-versions oauth.openshift.io/v1/OAuthClient --helm-api-versions oauth.openshift.io/v1/OAuthClientAuthorization --helm-api-versions oauth.openshift.io/v1/UserOAuthAccessToken --helm-api-versions operator.openshift.io/v1 --helm-api-versions operator.openshift.io/v1/CSISnapshotController --helm-api-versions operator.openshift.io/v1/CloudCredential --helm-api-versions operator.openshift.io/v1/ClusterCSIDriver --helm-api-versions operator.openshift.io/v1/Config --helm-api-versions operator.openshift.io/v1/Console --helm-api-versions operator.openshift.io/v1/DNS --helm-api-versions operator.openshift.io/v1/Etcd --helm-api-versions operator.openshift.io/v1/IngressController --helm-api-versions operator.openshift.io/v1/InsightsOperator --helm-api-versions operator.openshift.io/v1/KubeAPIServer --helm-api-versions operator.openshift.io/v1/KubeControllerManager --helm-api-versions operator.openshift.io/v1/KubeScheduler --helm-api-versions operator.openshift.io/v1/KubeStorageVersionMigrator --helm-api-versions operator.openshift.io/v1/MachineConfiguration --helm-api-versions operator.openshift.io/v1/Network --helm-api-versions operator.openshift.io/v1/OpenShiftAPIServer --helm-api-versions operator.openshift.io/v1/OpenShiftControllerManager --helm-api-versions operator.openshift.io/v1/ServiceCA --helm-api-versions operator.openshift.io/v1/Storage --helm-api-versions operator.openshift.io/v1alpha1 --helm-api-versions operator.openshift.io/v1alpha1/ImageContentSourcePolicy --helm-api-versions operators.coreos.com/v1 --helm-api-versions operators.coreos.com/v1/OLMConfig --helm-api-versions operators.coreos.com/v1/Operator --helm-api-versions operators.coreos.com/v1/OperatorCondition --helm-api-versions operators.coreos.com/v1/OperatorGroup --helm-api-versions operators.coreos.com/v1alpha1 --helm-api-versions operators.coreos.com/v1alpha1/CatalogSource --helm-api-versions operators.coreos.com/v1alpha1/ClusterServiceVersion --helm-api-versions operators.coreos.com/v1alpha1/InstallPlan --helm-api-versions operators.coreos.com/v1alpha1/Subscription --helm-api-versions operators.coreos.com/v1alpha2 --helm-api-versions operators.coreos.com/v1alpha2/OperatorGroup --helm-api-versions operators.coreos.com/v2 --helm-api-versions operators.coreos.com/v2/OperatorCondition --helm-api-versions package-operator.run/v1alpha1 --helm-api-versions package-operator.run/v1alpha1/ClusterObjectDeployment --helm-api-versions package-operator.run/v1alpha1/ClusterObjectSet --helm-api-versions package-operator.run/v1alpha1/ClusterObjectSetPhase --helm-api-versions package-operator.run/v1alpha1/ClusterObjectSlice --helm-api-versions package-operator.run/v1alpha1/ClusterObjectTemplate --helm-api-versions package-operator.run/v1alpha1/ClusterPackage --helm-api-versions package-operator.run/v1alpha1/HostedClusterPackage --helm-api-versions package-operator.run/v1alpha1/ObjectDeployment --helm-api-versions package-operator.run/v1alpha1/ObjectSet --helm-api-versions package-operator.run/v1alpha1/ObjectSetPhase --helm-api-versions package-operator.run/v1alpha1/ObjectSlice --helm-api-versions package-operator.run/v1alpha1/ObjectTemplate --helm-api-versions package-operator.run/v1alpha1/Package --helm-api-versions pipelines.openshift.io/v1alpha1 --helm-api-versions pipelines.openshift.io/v1alpha1/GitopsService --helm-api-versions policy.networking.k8s.io/v1alpha1 --helm-api-versions policy.networking.k8s.io/v1alpha1/AdminNetworkPolicy --helm-api-versions policy.networking.k8s.io/v1alpha1/BaselineAdminNetworkPolicy --helm-api-versions policy/v1 --helm-api-versions policy/v1/PodDisruptionBudget --helm-api-versions project.openshift.io/v1 --helm-api-versions project.openshift.io/v1/Project --helm-api-versions quota.openshift.io/v1 --helm-api-versions quota.openshift.io/v1/ClusterResourceQuota --helm-api-versions rbac.authorization.k8s.io/v1 --helm-api-versions rbac.authorization.k8s.io/v1/ClusterRole --helm-api-versions rbac.authorization.k8s.io/v1/ClusterRoleBinding --helm-api-versions rbac.authorization.k8s.io/v1/Role --helm-api-versions rbac.authorization.k8s.io/v1/RoleBinding --helm-api-versions route.openshift.io/v1 --helm-api-versions route.openshift.io/v1/Route --helm-api-versions samples.operator.openshift.io/v1 --helm-api-versions samples.operator.openshift.io/v1/Config --helm-api-versions scheduling.k8s.io/v1 --helm-api-versions scheduling.k8s.io/v1/PriorityClass --helm-api-versions security.internal.openshift.io/v1 --helm-api-versions security.internal.openshift.io/v1/RangeAllocation --helm-api-versions security.openshift.io/v1 --helm-api-versions security.openshift.io/v1/RangeAllocation --helm-api-versions security.openshift.io/v1/SecurityContextConstraints --helm-api-versions snapshot.storage.k8s.io/v1 --helm-api-versions snapshot.storage.k8s.io/v1/VolumeSnapshot --helm-api-versions snapshot.storage.k8s.io/v1/VolumeSnapshotClass --helm-api-versions snapshot.storage.k8s.io/v1/VolumeSnapshotContent --helm-api-versions storage.k8s.io/v1 --helm-api-versions storage.k8s.io/v1/CSIDriver --helm-api-versions storage.k8s.io/v1/CSINode --helm-api-versions storage.k8s.io/v1/CSIStorageCapacity --helm-api-versions storage.k8s.io/v1/StorageClass --helm-api-versions storage.k8s.io/v1/VolumeAttachment --helm-api-versions template.openshift.io/v1 --helm-api-versions template.openshift.io/v1/BrokerTemplateInstance --helm-api-versions template.openshift.io/v1/Template --helm-api-versions template.openshift.io/v1/TemplateInstance --helm-api-versions tuned.openshift.io/v1 --helm-api-versions tuned.openshift.io/v1/Profile --helm-api-versions tuned.openshift.io/v1/Tuned --helm-api-versions user.openshift.io/v1 --helm-api-versions user.openshift.io/v1/Group --helm-api-versions user.openshift.io/v1/Identity --helm-api-versions user.openshift.io/v1/User --helm-api-versions v1 --helm-api-versions v1/ConfigMap --helm-api-versions v1/Endpoints --helm-api-versions v1/Event --helm-api-versions v1/LimitRange --helm-api-versions v1/Namespace --helm-api-versions v1/Node --helm-api-versions v1/PersistentVolume --helm-api-versions v1/PersistentVolumeClaim --helm-api-versions v1/Pod --helm-api-versions v1/PodTemplate --helm-api-versions v1/ReplicationController --helm-api-versions v1/ResourceQuota --helm-api-versions v1/Secret --helm-api-versions v1/Service --helm-api-versions v1/ServiceAccount --helm-api-versions velero.io/v1 --helm-api-versions velero.io/v1/Backup --helm-api-versions velero.io/v1/BackupRepository --helm-api-versions velero.io/v1/BackupStorageLocation --helm-api-versions velero.io/v1/DeleteBackupRequest --helm-api-versions velero.io/v1/DownloadRequest --helm-api-versions velero.io/v1/PodVolumeBackup --helm-api-versions velero.io/v1/PodVolumeRestore --helm-api-versions velero.io/v1/Restore --helm-api-versions velero.io/v1/Schedule --helm-api-versions velero.io/v1/ServerStatusRequest --helm-api-versions velero.io/v1/VolumeSnapshotLocation --helm-api-versions velero.io/v2alpha1 --helm-api-versions velero.io/v2alpha1/DataDownload --helm-api-versions velero.io/v2alpha1/DataUpload --helm-api-versions whereabouts.cni.cncf.io/v1alpha1 --helm-api-versions whereabouts.cni.cncf.io/v1alpha1/IPPool --helm-api-versions whereabouts.cni.cncf.io/v1alpha1/NodeSlicePool --helm-api-versions whereabouts.cni.cncf.io/v1alpha1/OverlappingRangeIPReservation --helm-api-versions work.open-cluster-management.io/v1 --helm-api-versions work.open-cluster-management.io/v1/AppliedManifestWork` failed exit status 1: Error: Error: failed to untar: a file or directory with the name /components/vector-kubearchive-log-collector/development/charts/loki-6.49.0/loki-6.49.0.tgz already exists [2026-04-21 17:39:06] [WARN] │ └─ : unable to run: 'helm pull --untar --untardir /components/vector-kubearchive-log-collector/development/charts/loki-6.49.0 --repo https://grafana.github.io/helm-charts loki --version 6.49.0' with env=[HELM_CONFIG_HOME=/tmp/kustomize-helm-2501507588/helm HELM_CACHE_HOME=/tmp/kustomize-helm-2501507588/helm/.cache HELM_DATA_HOME=/tmp/kustomize-helm-2501507588/helm/.data] (is 'helm' installed?): exit status 1 [2026-04-21 17:39:06] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:39:16] [PROGRESS] Applications: 22/44 ready | 22 pending (3m 31s elapsed) [2026-04-21 17:39:16] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:39:16] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:39:26] [PROGRESS] Applications: 22/44 ready | 22 pending (3m 41s elapsed) [2026-04-21 17:39:26] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:39:26] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:39:36] [PROGRESS] Applications: 22/44 ready | 22 pending (3m 51s elapsed) [2026-04-21 17:39:36] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:39:36] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:39:46] [PROGRESS] Applications: 22/44 ready | 22 pending (4m 1s elapsed) [2026-04-21 17:39:46] [INFO] Pending: build-service-in-cluster-local,crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:39:46] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:39:57] [PROGRESS] Applications: 23/44 ready | 21 pending (4m 11s elapsed) [2026-04-21 17:39:57] [INFO] Pending: crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,konflux-rbac-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,monitoring-workload-prometheus-in-cluster-local,multi-platform-controller-in-cluster-local,perf-team-prometheus-reader-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:39:57] [SUBSTEP] Detailed status of pending applications: [2026-04-21 17:39:57] [INFO] ├─ App: crossplane-control-plane-in-cluster-local [2026-04-21 17:39:57] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:57] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:57] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:39:57] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:57] [WARN] │ │ └─ ClusterRole/namespace-claim-cleaner: Missing [2026-04-21 17:39:57] [WARN] │ │ └─ ClusterRole/provider-kubernetes-extra: Missing [2026-04-21 17:39:57] [INFO] │ └─ Message: No message [2026-04-21 17:39:57] [INFO] ├─ App: disaster-recovery-in-cluster-local [2026-04-21 17:39:57] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:57] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:57] [INFO] │ ├─ Out-of-sync resources: 11 [2026-04-21 17:39:57] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:57] [WARN] │ │ └─ Namespace/konflux-disaster-recovery: Missing [2026-04-21 17:39:57] [WARN] │ │ └─ ServiceAccount/cron-trigger: Missing [2026-04-21 17:39:57] [WARN] │ │ └─ CronJob/run-disaster-recovery-pipelinerun: Missing [2026-04-21 17:39:57] [INFO] │ └─ Message: No message [2026-04-21 17:39:57] [INFO] ├─ App: integration-in-cluster-local [2026-04-21 17:39:57] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:57] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:39:57] [INFO] │ ├─ Out-of-sync resources: 19 [2026-04-21 17:39:57] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:57] [WARN] │ │ └─ CustomResourceDefinition/componentgroups.appstudio.redhat.com: Missing [2026-04-21 17:39:57] [WARN] │ │ └─ CustomResourceDefinition/integrationtestscenarios.appstudio.redhat.com: Missing [2026-04-21 17:39:57] [WARN] │ │ └─ ClusterRole/crd-manager-for-integration: Missing [2026-04-21 17:39:57] [INFO] │ └─ Message: No message [2026-04-21 17:39:57] [INFO] ├─ App: knative-eventing-in-cluster-local [2026-04-21 17:39:57] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:57] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:39:57] [INFO] │ ├─ Out-of-sync resources: 14 [2026-04-21 17:39:58] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:58] [WARN] │ │ └─ CustomResourceDefinition/apiserversources.sources.knative.dev: Missing [2026-04-21 17:39:58] [WARN] │ │ └─ CustomResourceDefinition/brokers.eventing.knative.dev: Missing [2026-04-21 17:39:58] [WARN] │ │ └─ CustomResourceDefinition/channels.messaging.knative.dev: Missing [2026-04-21 17:39:58] [INFO] │ └─ Message: No message [2026-04-21 17:39:58] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-04-21 17:39:58] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:58] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:58] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:39:58] [INFO] │ └─ Message: No message [2026-04-21 17:39:58] [INFO] ├─ App: konflux-rbac-in-cluster-local [2026-04-21 17:39:58] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:58] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:58] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:39:58] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:58] [WARN] │ │ └─ ClusterRole/appstudio-pipelines-runner: Missing [2026-04-21 17:39:58] [WARN] │ │ └─ ClusterRole/konflux-integration-runner: Missing [2026-04-21 17:39:58] [INFO] │ └─ Message: No message [2026-04-21 17:39:58] [INFO] ├─ App: kubearchive-in-cluster-local [2026-04-21 17:39:58] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:58] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:39:58] [INFO] │ ├─ Out-of-sync resources: 13 [2026-04-21 17:39:58] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:58] [WARN] │ │ └─ CustomResourceDefinition/clusterkubearchiveconfigs.kubearchive.org: Missing [2026-04-21 17:39:58] [WARN] │ │ └─ CustomResourceDefinition/clustervacuumconfigs.kubearchive.org: Missing [2026-04-21 17:39:58] [WARN] │ │ └─ CustomResourceDefinition/kubearchiveconfigs.kubearchive.org: Missing [2026-04-21 17:39:58] [INFO] │ └─ Message: No message [2026-04-21 17:39:58] [INFO] ├─ App: kueue-in-cluster-local [2026-04-21 17:39:58] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:58] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:58] [INFO] │ ├─ Out-of-sync resources: 8 [2026-04-21 17:39:58] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:58] [WARN] │ │ └─ ServiceMonitor/tekton-kueue-controller: Missing [2026-04-21 17:39:58] [WARN] │ │ └─ ServiceMonitor/tekton-kueue-webhook: Missing [2026-04-21 17:39:58] [WARN] │ │ └─ ClusterRole/alert-mgr-kueue-admission-manager-role: Missing [2026-04-21 17:39:58] [INFO] │ └─ Message: No message [2026-04-21 17:39:59] [INFO] ├─ App: kyverno-in-cluster-local [2026-04-21 17:39:59] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:59] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:39:59] [INFO] │ ├─ Out-of-sync resources: 31 [2026-04-21 17:39:59] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:59] [WARN] │ │ └─ CustomResourceDefinition/cleanuppolicies.kyverno.io: Missing [2026-04-21 17:39:59] [WARN] │ │ └─ CustomResourceDefinition/clustercleanuppolicies.kyverno.io: Missing [2026-04-21 17:39:59] [WARN] │ │ └─ CustomResourceDefinition/clusterephemeralreports.reports.kyverno.io: Missing [2026-04-21 17:39:59] [INFO] │ └─ Message: No message [2026-04-21 17:39:59] [INFO] ├─ App: monitoring-registry-in-cluster-local [2026-04-21 17:39:59] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:59] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:59] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:39:59] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:59] [WARN] │ │ └─ ClusterRole/kube-rbac-proxy-registry: Missing [2026-04-21 17:39:59] [WARN] │ │ └─ ClusterRole/registry-exporter-role-metrics-reader: Missing [2026-04-21 17:39:59] [INFO] │ └─ Message: No message [2026-04-21 17:39:59] [INFO] ├─ App: monitoring-workload-prometheus-in-cluster-local [2026-04-21 17:39:59] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:59] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:59] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:39:59] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:59] [WARN] │ │ └─ ClusterRole/kube-rbac-proxy: Missing [2026-04-21 17:39:59] [WARN] │ │ └─ ClusterRole/o11y-dummy-service-metrics-reader: Missing [2026-04-21 17:39:59] [INFO] │ └─ Message: No message [2026-04-21 17:39:59] [INFO] ├─ App: multi-platform-controller-in-cluster-local [2026-04-21 17:39:59] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:39:59] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:39:59] [INFO] │ ├─ Out-of-sync resources: 3 [2026-04-21 17:39:59] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:39:59] [WARN] │ │ └─ ClusterRole/multi-platform-controller-manager-role: Missing [2026-04-21 17:39:59] [WARN] │ │ └─ ClusterRole/multi-platform-controller-metrics-auth-role: Missing [2026-04-21 17:39:59] [WARN] │ │ └─ ClusterRole/multi-platform-controller-metrics-reader: Missing [2026-04-21 17:39:59] [INFO] │ └─ Message: No message [2026-04-21 17:40:00] [INFO] ├─ App: perf-team-prometheus-reader-in-cluster-local [2026-04-21 17:40:00] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:00] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:40:00] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:40:00] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:00] [WARN] │ │ └─ ClusterRole/perf-team-prometheus-reader-oomcrash-role: Missing [2026-04-21 17:40:00] [INFO] │ └─ Message: No message [2026-04-21 17:40:00] [INFO] ├─ App: pipeline-service-in-cluster-local [2026-04-21 17:40:00] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:00] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:40:00] [INFO] │ ├─ Out-of-sync resources: 23 [2026-04-21 17:40:00] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:00] [WARN] │ │ └─ Service/tekton-results-watcher: Missing [2026-04-21 17:40:00] [WARN] │ │ └─ CustomResourceDefinition/miniojobs.job.min.io: Missing [2026-04-21 17:40:00] [WARN] │ │ └─ CustomResourceDefinition/policybindings.sts.min.io: Missing [2026-04-21 17:40:00] [INFO] │ └─ Message: No message [2026-04-21 17:40:00] [INFO] ├─ App: policies-in-cluster-local [2026-04-21 17:40:00] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:00] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:40:00] [INFO] │ ├─ Out-of-sync resources: 34 [2026-04-21 17:40:00] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:00] [WARN] │ │ └─ ValidatingAdmissionPolicy/no-skip.kueue.konflux-ci.dev: Missing [2026-04-21 17:40:00] [WARN] │ │ └─ ValidatingAdmissionPolicyBinding/no-skip.kueue.konflux-ci.dev: Missing [2026-04-21 17:40:00] [WARN] │ │ └─ ClusterPolicy/bootstrap-tenant-namespace-queue: Missing [2026-04-21 17:40:00] [INFO] │ └─ Message: No message [2026-04-21 17:40:00] [INFO] ├─ App: project-controller-in-cluster-local [2026-04-21 17:40:00] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:00] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:40:00] [INFO] │ ├─ Out-of-sync resources: 3 [2026-04-21 17:40:00] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:00] [WARN] │ │ └─ CustomResourceDefinition/projectdevelopmentstreams.projctl.konflux.dev: Missing [2026-04-21 17:40:00] [WARN] │ │ └─ CustomResourceDefinition/projectdevelopmentstreamtemplates.projctl.konflux.dev: Missing [2026-04-21 17:40:00] [WARN] │ │ └─ CustomResourceDefinition/projects.projctl.konflux.dev: Missing [2026-04-21 17:40:00] [INFO] │ └─ Message: No message [2026-04-21 17:40:01] [INFO] ├─ App: release-in-cluster-local [2026-04-21 17:40:01] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:01] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:40:01] [INFO] │ ├─ Out-of-sync resources: 26 [2026-04-21 17:40:01] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:01] [WARN] │ │ └─ CustomResourceDefinition/releaseplanadmissions.appstudio.redhat.com: Missing [2026-04-21 17:40:01] [WARN] │ │ └─ CustomResourceDefinition/releaseplans.appstudio.redhat.com: Missing [2026-04-21 17:40:01] [WARN] │ │ └─ CustomResourceDefinition/releases.appstudio.redhat.com: Missing [2026-04-21 17:40:01] [INFO] │ └─ Message: No message [2026-04-21 17:40:01] [INFO] ├─ App: tempo-in-cluster-local [2026-04-21 17:40:01] [INFO] │ ├─ Sync Status: Synced [2026-04-21 17:40:01] [INFO] │ ├─ Health Status: Degraded [2026-04-21 17:40:01] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:01] [WARN] │ │ └─ Subscription/tempo-product: Degraded [2026-04-21 17:40:01] [INFO] │ └─ Message: No message [2026-04-21 17:40:01] [INFO] ├─ App: trust-manager-in-cluster-local [2026-04-21 17:40:01] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:01] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:40:01] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:40:01] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:01] [WARN] │ │ └─ CustomResourceDefinition/bundles.trust.cert-manager.io: Missing [2026-04-21 17:40:01] [WARN] │ │ └─ ClusterRole/trust-manager: Missing [2026-04-21 17:40:01] [INFO] │ └─ Message: No message [2026-04-21 17:40:01] [INFO] ├─ App: vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:40:01] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:01] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:40:01] [INFO] │ ├─ Out-of-sync resources: 53 [2026-04-21 17:40:01] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:01] [WARN] │ │ └─ ConfigMap/grafana: Missing [2026-04-21 17:40:01] [WARN] │ │ └─ ConfigMap/loki: Missing [2026-04-21 17:40:01] [WARN] │ │ └─ ConfigMap/loki-gateway: Missing [2026-04-21 17:40:01] [WARN] │ └─ Conditions: [2026-04-21 17:40:01] [WARN] │ └─ [RepeatedResourceWarning] Resource /ServiceAccount/product-kubearchive-logging/minio-sa appeared 2 times among application resources. [2026-04-21 17:40:02] [INFO] ├─ App: vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:40:02] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:02] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:40:02] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:40:02] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:02] [WARN] │ │ └─ ClusterRole/vector-tekton-logs-collector: Missing [2026-04-21 17:40:02] [INFO] │ └─ Message: No message [2026-04-21 17:40:02] [WARN] Found applications in Unknown state (not Progressing), investigating... [2026-04-21 17:40:02] [ERROR] Application 'vector-kubearchive-log-collector-in-cluster-local' is in Unknown state without 'context deadline exceeded' [2026-04-21 17:40:02] [INFO] ├─ App: vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:40:02] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:40:02] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:40:02] [INFO] │ ├─ Out-of-sync resources: 53 [2026-04-21 17:40:02] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:40:02] [WARN] │ │ └─ ConfigMap/grafana: Missing [2026-04-21 17:40:02] [WARN] │ │ └─ ConfigMap/loki: Missing [2026-04-21 17:40:02] [WARN] │ │ └─ ConfigMap/loki-gateway: Missing [2026-04-21 17:40:02] [WARN] │ └─ Conditions: [2026-04-21 17:40:02] [WARN] │ └─ [RepeatedResourceWarning] Resource /ServiceAccount/product-kubearchive-logging/minio-sa appeared 2 times among application resources. [2026-04-21 17:40:02] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:40:12] [PROGRESS] Applications: 26/44 ready | 18 pending (4m 27s elapsed) [2026-04-21 17:40:12] [INFO] Pending: crossplane-control-plane-in-cluster-local,disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,monitoring-registry-in-cluster-local,multi-platform-controller-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local,vector-tekton-logs-collector-in-cluster-local [2026-04-21 17:40:12] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:40:22] [PROGRESS] Applications: 30/44 ready | 14 pending (4m 37s elapsed) [2026-04-21 17:40:22] [INFO] Pending: disaster-recovery-in-cluster-local,integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,project-controller-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:40:22] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:40:33] [PROGRESS] Applications: 32/44 ready | 12 pending (4m 47s elapsed) [2026-04-21 17:40:33] [INFO] Pending: integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,pipeline-service-in-cluster-local,policies-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:40:33] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:40:43] [PROGRESS] Applications: 33/44 ready | 11 pending (4m 58s elapsed) [2026-04-21 17:40:43] [INFO] Pending: integration-in-cluster-local,knative-eventing-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,pipeline-service-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:40:43] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:40:53] [PROGRESS] Applications: 34/44 ready | 10 pending (5m 8s elapsed) [2026-04-21 17:40:53] [INFO] Pending: integration-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,kyverno-in-cluster-local,pipeline-service-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:40:53] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:41:03] [PROGRESS] Applications: 35/44 ready | 9 pending (5m 18s elapsed) [2026-04-21 17:41:03] [INFO] Pending: integration-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,kueue-in-cluster-local,pipeline-service-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local,vector-kubearchive-log-collector-in-cluster-local [2026-04-21 17:41:03] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:41:13] [PROGRESS] Applications: 37/44 ready | 7 pending (5m 28s elapsed) [2026-04-21 17:41:13] [INFO] Pending: integration-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,release-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:41:13] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:41:24] [PROGRESS] Applications: 38/44 ready | 6 pending (5m 38s elapsed) [2026-04-21 17:41:24] [INFO] Pending: integration-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:41:24] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:41:34] [PROGRESS] Applications: 38/44 ready | 6 pending (5m 49s elapsed) [2026-04-21 17:41:34] [INFO] Pending: integration-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:41:34] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:41:44] [PROGRESS] Applications: 38/44 ready | 6 pending (5m 59s elapsed) [2026-04-21 17:41:44] [INFO] Pending: integration-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:41:44] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:41:54] [PROGRESS] Applications: 38/44 ready | 6 pending (6m 9s elapsed) [2026-04-21 17:41:54] [INFO] Pending: integration-in-cluster-local,konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:41:54] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:42:04] [PROGRESS] Applications: 39/44 ready | 5 pending (6m 19s elapsed) [2026-04-21 17:42:04] [INFO] Pending: konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:42:04] [SUBSTEP] Detailed status of pending applications: [2026-04-21 17:42:04] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-04-21 17:42:04] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:42:04] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:42:05] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:42:05] [INFO] │ └─ Message: No message [2026-04-21 17:42:05] [INFO] ├─ App: kubearchive-in-cluster-local [2026-04-21 17:42:05] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:42:05] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:42:05] [INFO] │ ├─ Out-of-sync resources: 5 [2026-04-21 17:42:05] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:42:05] [WARN] │ │ └─ CustomResourceDefinition/clusterkubearchiveconfigs.kubearchive.org: Missing [2026-04-21 17:42:05] [WARN] │ │ └─ CustomResourceDefinition/clustervacuumconfigs.kubearchive.org: Missing [2026-04-21 17:42:05] [WARN] │ │ └─ CustomResourceDefinition/kubearchiveconfigs.kubearchive.org: Missing [2026-04-21 17:42:05] [INFO] │ └─ Message: No message [2026-04-21 17:42:05] [INFO] ├─ App: pipeline-service-in-cluster-local [2026-04-21 17:42:05] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:42:05] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:42:05] [INFO] │ ├─ Out-of-sync resources: 2 [2026-04-21 17:42:05] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:42:05] [WARN] │ │ └─ Service/tekton-results-watcher: Missing [2026-04-21 17:42:05] [WARN] │ │ └─ Deployment/tekton-results-watcher: Missing [2026-04-21 17:42:05] [INFO] │ └─ Message: No message [2026-04-21 17:42:05] [INFO] ├─ App: tempo-in-cluster-local [2026-04-21 17:42:05] [INFO] │ ├─ Sync Status: Synced [2026-04-21 17:42:05] [INFO] │ ├─ Health Status: Degraded [2026-04-21 17:42:05] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:42:05] [WARN] │ │ └─ Subscription/tempo-product: Degraded [2026-04-21 17:42:05] [INFO] │ └─ Message: No message [2026-04-21 17:42:05] [INFO] ├─ App: trust-manager-in-cluster-local [2026-04-21 17:42:05] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:42:05] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:42:05] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:42:05] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:42:05] [WARN] │ │ └─ CustomResourceDefinition/bundles.trust.cert-manager.io: Missing [2026-04-21 17:42:05] [INFO] │ └─ Message: No message [2026-04-21 17:42:06] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:42:16] [PROGRESS] Applications: 39/44 ready | 5 pending (6m 31s elapsed) [2026-04-21 17:42:16] [INFO] Pending: konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:42:16] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:42:26] [PROGRESS] Applications: 39/44 ready | 5 pending (6m 41s elapsed) [2026-04-21 17:42:26] [INFO] Pending: konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:42:26] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:42:36] [PROGRESS] Applications: 39/44 ready | 5 pending (6m 51s elapsed) [2026-04-21 17:42:36] [INFO] Pending: konflux-kite-in-cluster-local,kubearchive-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:42:36] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:42:46] [PROGRESS] Applications: 40/44 ready | 4 pending (7m 1s elapsed) [2026-04-21 17:42:46] [INFO] Pending: konflux-kite-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:42:46] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:42:56] [PROGRESS] Applications: 40/44 ready | 4 pending (7m 11s elapsed) [2026-04-21 17:42:56] [INFO] Pending: konflux-kite-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:42:56] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:43:07] [PROGRESS] Applications: 40/44 ready | 4 pending (7m 21s elapsed) [2026-04-21 17:43:07] [INFO] Pending: konflux-kite-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:43:07] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:43:17] [PROGRESS] Applications: 40/44 ready | 4 pending (7m 32s elapsed) [2026-04-21 17:43:17] [INFO] Pending: konflux-kite-in-cluster-local,pipeline-service-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:43:17] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:43:27] [PROGRESS] Applications: 41/44 ready | 3 pending (7m 42s elapsed) [2026-04-21 17:43:27] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:43:27] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:43:37] [PROGRESS] Applications: 41/44 ready | 3 pending (7m 52s elapsed) [2026-04-21 17:43:37] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:43:37] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:43:47] [PROGRESS] Applications: 42/44 ready | 2 pending (8m 2s elapsed) [2026-04-21 17:43:47] [INFO] Pending: tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:43:47] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:43:58] [PROGRESS] Applications: 41/44 ready | 3 pending (8m 12s elapsed) [2026-04-21 17:43:58] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:43:58] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:44:08] [PROGRESS] Applications: 41/44 ready | 3 pending (8m 23s elapsed) [2026-04-21 17:44:08] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:44:08] [SUBSTEP] Detailed status of pending applications: [2026-04-21 17:44:08] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-04-21 17:44:08] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:44:08] [INFO] │ ├─ Health Status: Progressing [2026-04-21 17:44:08] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:44:08] [INFO] │ └─ Message: No message [2026-04-21 17:44:08] [INFO] ├─ App: tempo-in-cluster-local [2026-04-21 17:44:08] [INFO] │ ├─ Sync Status: Synced [2026-04-21 17:44:08] [INFO] │ ├─ Health Status: Degraded [2026-04-21 17:44:08] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:44:08] [WARN] │ │ └─ Subscription/tempo-product: Degraded [2026-04-21 17:44:08] [INFO] │ └─ Message: No message [2026-04-21 17:44:09] [INFO] ├─ App: trust-manager-in-cluster-local [2026-04-21 17:44:09] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:44:09] [INFO] │ ├─ Health Status: Missing [2026-04-21 17:44:09] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:44:09] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:44:09] [WARN] │ │ └─ CustomResourceDefinition/bundles.trust.cert-manager.io: Missing [2026-04-21 17:44:09] [INFO] │ └─ Message: No message [2026-04-21 17:44:09] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:44:19] [PROGRESS] Applications: 41/44 ready | 3 pending (8m 34s elapsed) [2026-04-21 17:44:19] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:44:19] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:44:29] [PROGRESS] Applications: 41/44 ready | 3 pending (8m 44s elapsed) [2026-04-21 17:44:29] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local,trust-manager-in-cluster-local [2026-04-21 17:44:29] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:44:39] [PROGRESS] Applications: 42/44 ready | 2 pending (8m 54s elapsed) [2026-04-21 17:44:39] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:44:39] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:44:49] [PROGRESS] Applications: 42/44 ready | 2 pending (9m 4s elapsed) [2026-04-21 17:44:49] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:44:49] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:44:59] [PROGRESS] Applications: 42/44 ready | 2 pending (9m 14s elapsed) [2026-04-21 17:44:59] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:44:59] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:45:10] [PROGRESS] Applications: 42/44 ready | 2 pending (9m 24s elapsed) [2026-04-21 17:45:10] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:45:10] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:45:20] [PROGRESS] Applications: 42/44 ready | 2 pending (9m 35s elapsed) [2026-04-21 17:45:20] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:45:20] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:45:30] [PROGRESS] Applications: 42/44 ready | 2 pending (9m 45s elapsed) [2026-04-21 17:45:30] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:45:30] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:45:40] [PROGRESS] Applications: 42/44 ready | 2 pending (9m 55s elapsed) [2026-04-21 17:45:40] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:45:40] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:45:50] [PROGRESS] Applications: 42/44 ready | 2 pending (10m 5s elapsed) [2026-04-21 17:45:50] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:45:50] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:46:01] [PROGRESS] Applications: 42/44 ready | 2 pending (10m 15s elapsed) [2026-04-21 17:46:01] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:46:01] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:46:11] [PROGRESS] Applications: 42/44 ready | 2 pending (10m 26s elapsed) [2026-04-21 17:46:11] [INFO] Pending: konflux-kite-in-cluster-local,tempo-in-cluster-local [2026-04-21 17:46:11] [SUBSTEP] Detailed status of pending applications: [2026-04-21 17:46:11] [INFO] ├─ App: konflux-kite-in-cluster-local [2026-04-21 17:46:11] [INFO] │ ├─ Sync Status: OutOfSync [2026-04-21 17:46:11] [INFO] │ ├─ Health Status: Healthy [2026-04-21 17:46:11] [INFO] │ ├─ Out-of-sync resources: 1 [2026-04-21 17:46:11] [INFO] │ └─ Message: No message [2026-04-21 17:46:11] [INFO] ├─ App: tempo-in-cluster-local [2026-04-21 17:46:11] [INFO] │ ├─ Sync Status: Synced [2026-04-21 17:46:11] [INFO] │ ├─ Health Status: Degraded [2026-04-21 17:46:11] [WARN] │ ├─ Degraded/Missing resources: [2026-04-21 17:46:11] [WARN] │ │ └─ Subscription/tempo-product: Degraded [2026-04-21 17:46:11] [INFO] │ └─ Message: No message [2026-04-21 17:46:11] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:46:22] [PROGRESS] Applications: 43/44 ready | 1 pending (10m 36s elapsed) [2026-04-21 17:46:22] [INFO] Pending: konflux-kite-in-cluster-local [2026-04-21 17:46:22] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:46:32] [PROGRESS] Applications: 43/44 ready | 1 pending (10m 47s elapsed) [2026-04-21 17:46:32] [INFO] Pending: konflux-kite-in-cluster-local [2026-04-21 17:46:32] [WAITING] Waiting 10 seconds before next sync check... [2026-04-21 17:46:42] [PROGRESS] Applications: 44/44 ready | 0 pending (10m 57s elapsed) [2026-04-21 17:46:42] [SUCCESS] All 44 ArgoCD applications are Synced and Healthy in 10m 57s ============================================================================= [2026-04-21 17:46:42] [STEP] Waiting for Tekton components to be ready ============================================================================= [2026-04-21 17:46:42] [INFO] Reference: https://tekton.dev/docs/operator/tektonconfig/#tekton-config [2026-04-21 17:46:42] [INFO] Timeout: 900 seconds (15 minutes) [2026-04-21 17:46:42] [PROGRESS] Tekton readiness check iteration 1: status=True (0m 0s elapsed) [2026-04-21 17:46:42] [SUCCESS] All Tekton components are installed and ready in 0m 0s ============================================================================= [2026-04-21 17:46:42] [STEP] Waiting for Tekton CRDs to be available ============================================================================= [2026-04-21 17:46:42] [PROGRESS] Tekton CRD check attempt 1/5 [2026-04-21 17:46:42] [SUCCESS] All required Tekton CRDs are available: pipelines tasks pipelineruns taskruns ============================================================================= [2026-04-21 17:46:42] [STEP] Configuring Pipelines as Code integration ============================================================================= [2026-04-21 17:46:42] [INFO] ============================================================================= [2026-04-21 17:46:42] [INFO] Pipelines as Code (PAC) Integration Setup [2026-04-21 17:46:42] [INFO] ============================================================================= [2026-04-21 17:46:42] [INFO] PAC Namespace: openshift-pipelines [2026-04-21 17:46:42] [INFO] PAC Secret Name: pipelines-as-code-secret [2026-04-21 17:46:42] [INFO] Integration Namespace: integration-service [2026-04-21 17:46:42] [INFO] Authentication method: GitHub App [2026-04-21 17:46:42] [INFO] - GitHub App ID: 310332 [2026-04-21 17:46:42] [INFO] Using existing QE sprayproxy configuration (webhook secret provided) [2026-04-21 17:46:42] [SUCCESS] GitHub App credentials configured [2026-04-21 17:46:42] [INFO] GitHub token provided via MY_GITHUB_TOKEN (fallback) [2026-04-21 17:46:42] [DEBUG] No GitLab token provided (PAC_GITLAB_TOKEN not set) [2026-04-21 17:46:42] [INFO] Creating required namespaces [2026-04-21 17:46:42] [SUBSTEP] Ensuring namespace 'openshift-pipelines' exists [2026-04-21 17:46:42] [DEBUG] Namespace 'openshift-pipelines' already exists [2026-04-21 17:46:42] [SUBSTEP] Ensuring namespace 'build-service' exists [2026-04-21 17:46:43] [DEBUG] Namespace 'build-service' already exists [2026-04-21 17:46:43] [SUBSTEP] Ensuring namespace 'integration-service' exists [2026-04-21 17:46:43] [DEBUG] Namespace 'integration-service' already exists [2026-04-21 17:46:43] [INFO] Configuring PAC secrets across namespaces [2026-04-21 17:46:43] [SUBSTEP] Creating PAC secret in namespace 'openshift-pipelines' secret/pipelines-as-code-secret created [2026-04-21 17:46:44] [SUCCESS] PAC secret configured in 'openshift-pipelines' [2026-04-21 17:46:44] [SUBSTEP] Creating PAC secret in namespace 'build-service' secret/pipelines-as-code-secret created [2026-04-21 17:46:44] [SUCCESS] PAC secret configured in 'build-service' [2026-04-21 17:46:44] [SUBSTEP] Creating PAC secret in namespace 'integration-service' secret/pipelines-as-code-secret created [2026-04-21 17:46:45] [SUCCESS] PAC secret configured in 'integration-service' [2026-04-21 17:46:45] [SUBSTEP] Creating PAC secret in namespace 'mintmaker' secret/pipelines-as-code-secret created [2026-04-21 17:46:45] [SUCCESS] PAC secret configured in 'mintmaker' [2026-04-21 17:46:45] [INFO] ============================================================================= [2026-04-21 17:46:45] [SUCCESS] PAC Integration Setup Complete [2026-04-21 17:46:45] [INFO] ============================================================================= [2026-04-21 17:46:45] [INFO] Configured namespaces: [2026-04-21 17:46:45] [INFO] - openshift-pipelines (PAC controller) [2026-04-21 17:46:45] [INFO] - build-service (Build Service) [2026-04-21 17:46:45] [INFO] - integration-service (Integration Service) [2026-04-21 17:46:45] [INFO] - mintmaker (Mintmaker) [PAC_SETUP_JSON] {"status":"success","namespaces":["openshift-pipelines","build-service","integration-service","mintmaker"],"github_app_configured":true,"github_token_configured":true,"gitlab_token_configured":false} [2026-04-21 17:46:45] [SUCCESS] Pipelines as Code configured ============================================================================= [2026-04-21 17:46:45] [STEP] Preview Environment Setup Complete ============================================================================= [2026-04-21 17:46:45] [SUCCESS] Konflux preview environment is ready! [2026-04-21 17:46:45] [INFO] - Fork: https://github.com/redhat-appstudio-qe/infra-deployments.git [2026-04-21 17:46:45] [INFO] - Branch: preview-main-swve [2026-04-21 17:46:45] [INFO] - GitHub Org: redhat-appstudio-qe [2026-04-21 17:46:45] [INFO] - OpenShift Version: 4.18.9 [2026-04-21 17:46:45] [INFO] - End time: 2026-04-21 17:46:45 UTC ============================================================================= [2026-04-21 17:46:45] [STEP] Execution Summary ============================================================================= [2026-04-21 17:46:45] [SUCCESS] Status: SUCCESS [2026-04-21 17:46:45] [INFO] Total Execution Time: 13m 13s (793 seconds) [2026-04-21 17:46:45] [INFO] Applications Deployed: 44 [EXECUTION_SUMMARY_JSON] {"status":"success","total_time_seconds":793,"apps_deployed":44,"ocp_version":"4.18.9","preview_branch":"preview-main-swve","git_repo":"https://github.com/redhat-appstudio-qe/infra-deployments.git"} Switched to branch 'main' Your branch is up to date with 'upstream/main'. ============================================================================= [2026-04-21 17:46:45] [STEP] Cluster Bootstrap Complete ============================================================================= [2026-04-21 17:46:45] [SUCCESS] Konflux cluster bootstrap finished successfully [2026-04-21 17:46:45] [INFO] - Mode: preview [2026-04-21 17:46:45] [INFO] - Total time: 15m 2s [2026-04-21 17:46:45] [INFO] - End time: 2026-04-21 17:46:45 UTC [BOOTSTRAP_SUMMARY_JSON] {"status":"success","mode":"preview","obo_enabled":false,"eaas_enabled":false,"total_time_seconds":902} [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed. Detected at: > goroutine 116 [running]: > runtime/debug.Stack() > /usr/lib/golang/src/runtime/debug/stack.go:26 +0x5e > sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot() > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/log/log.go:60 +0xcd > sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).WithName(0xc00094fc40, {0x375d748, 0x14}) > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/log/deleg.go:147 +0x3e > github.com/go-logr/logr.Logger.WithName({{0x3fd9000, 0xc00094fc40}, 0x0}, {0x375d748?, 0x0?}) > /opt/app-root/src/go/pkg/mod/github.com/go-logr/logr@v1.4.3/logr.go:345 +0x36 > sigs.k8s.io/controller-runtime/pkg/client.newClient(0x1?, {0x0, 0xc000847ea0, {0x0, 0x0}, 0x0, 0x0}) > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/client/client.go:118 +0xdb > sigs.k8s.io/controller-runtime/pkg/client.New(0xc00098e248?, {0x0, 0xc000847ea0, {0x0, 0x0}, 0x0, 0x0}) > /opt/app-root/src/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.2/pkg/client/client.go:98 +0x4d > github.com/konflux-ci/e2e-tests/pkg/clients/kubernetes.NewAdminKubernetesClient() > /tmp/tmp.dle8pz85Yc/pkg/clients/kubernetes/client.go:159 +0x8f > github.com/konflux-ci/e2e-tests/pkg/clients/sprayproxy.GetPaCHost() > /tmp/tmp.dle8pz85Yc/pkg/clients/sprayproxy/sprayproxy.go:93 +0x1c > github.com/konflux-ci/e2e-tests/magefiles/rulesengine/repos.registerPacServer() > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/repos/common.go:421 +0x78 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine/repos.init.func8(0xc000bad908?) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/repos/common.go:373 +0x25 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.ActionFunc.Execute(0xc?, 0x3734df1?) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:279 +0x19 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Apply(...) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:315 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Check(0x61267a0, 0xc000bad908) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:348 +0xb3 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.All.Check({0x611e6e0?, 0xc001803bd8?, 0x2409219?}, 0xc000bad908) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:245 +0x4f > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Eval(...) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:308 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Check(0x6126860, 0xc000bad908) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:340 +0x2b > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.All.Check({0x61284e0?, 0x1a0?, 0x0?}, 0xc000bad908) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:245 +0x4f > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*Rule).Eval(...) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:308 > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*RuleEngine).runLoadedCatalog(0x615f048, {0xc000ee1188?, 0xc000cf5e60?, 0x47?}, 0xc000bad908) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:129 +0x11f > github.com/konflux-ci/e2e-tests/magefiles/rulesengine.(*RuleEngine).RunRulesOfCategory(0x615f048, {0x372eda2, 0x2}, 0xc000bad908) > /tmp/tmp.dle8pz85Yc/magefiles/rulesengine/types.go:121 +0x1b4 > main.CI.TestE2E({}) > /tmp/tmp.dle8pz85Yc/magefiles/magefile.go:331 +0x18a > main.main.func20({0xc000994b60?, 0x0?}) > /tmp/tmp.dle8pz85Yc/magefiles/mage_output_file.go:888 +0xf > main.main.func12.1() > /tmp/tmp.dle8pz85Yc/magefiles/mage_output_file.go:306 +0x5b > created by main.main.func12 in goroutine 1 > /tmp/tmp.dle8pz85Yc/magefiles/mage_output_file.go:301 +0xbe I0421 17:46:46.127258 19172 common.go:429] Registered PaC server: https://pipelines-as-code-controller-openshift-pipelines.apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com I0421 17:46:46.199388 19172 common.go:454] The PaC servers registered in Sprayproxy: https://pipelines-as-code-controller-openshift-pipelines.apps.rosa.kx-283ab81d79.70f7.p3.openshiftapps.com, https://pipelines-as-code-controller-openshift-pipelines.apps.rosa.kx-4310c8a077.2y37.p3.openshiftapps.com I0421 17:46:46.199441 19172 common.go:470] going to create new Tekton bundle remote-build for the purpose of testing multi-platform-controller PR I0421 17:46:46.526225 19172 common.go:511] Found current task ref quay.io/konflux-ci/tekton-catalog/task-buildah:0.9@sha256:62f09c50d552eac57e17638c67e88b0982352a71975858c8ba262bcff293de06 I0421 17:46:46.528137 19172 util.go:512] found credentials for image ref quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793606-mxyb -> user: redhat-appstudio-qe+redhat_appstudio_quality Creating Tekton Bundle: - Added Pipeline: buildah-remote-pipeline to image I0421 17:46:47.809931 19172 bundle.go:57] image digest for a new tekton bundle quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793606-mxyb: quay.io/redhat-appstudio-qe/test-images@sha256:fd3a598e8ae8b0ae8d0cef58ded7e70bf1888a1f57611307a9ead00d9c9532e6 I0421 17:46:47.809953 19172 common.go:537] SETTING ENV VAR CUSTOM_BUILDAH_REMOTE_PIPELINE_BUILD_BUNDLE_ARM64 to value quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793606-mxyb I0421 17:46:48.075018 19172 common.go:511] Found current task ref quay.io/konflux-ci/tekton-catalog/task-buildah:0.9@sha256:62f09c50d552eac57e17638c67e88b0982352a71975858c8ba262bcff293de06 I0421 17:46:48.076638 19172 util.go:512] found credentials for image ref quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793607-ukga -> user: redhat-appstudio-qe+redhat_appstudio_quality Creating Tekton Bundle: - Added Pipeline: buildah-remote-pipeline to image I0421 17:46:49.446736 19172 bundle.go:57] image digest for a new tekton bundle quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793607-ukga: quay.io/redhat-appstudio-qe/test-images@sha256:b7c03fd05c89b52ddbf053b10bdf39c00ffd1b889def971ed31d675c42e0e5e5 I0421 17:46:49.446762 19172 common.go:537] SETTING ENV VAR CUSTOM_BUILDAH_REMOTE_PIPELINE_BUILD_BUNDLE_S390X to value quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793607-ukga I0421 17:46:49.701188 19172 common.go:511] Found current task ref quay.io/konflux-ci/tekton-catalog/task-buildah:0.9@sha256:62f09c50d552eac57e17638c67e88b0982352a71975858c8ba262bcff293de06 I0421 17:46:49.702827 19172 util.go:512] found credentials for image ref quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793609-egby -> user: redhat-appstudio-qe+redhat_appstudio_quality Creating Tekton Bundle: - Added Pipeline: buildah-remote-pipeline to image I0421 17:46:51.190050 19172 bundle.go:57] image digest for a new tekton bundle quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793609-egby: quay.io/redhat-appstudio-qe/test-images@sha256:d2eba4bb54a82d1aaaaf3860fad92bd041a9c11815a348ed533e51d736e6609d I0421 17:46:51.190073 19172 common.go:537] SETTING ENV VAR CUSTOM_BUILDAH_REMOTE_PIPELINE_BUILD_BUNDLE_PPC64LE to value quay.io/redhat-appstudio-qe/test-images:pipeline-bundle-1776793609-egby exec: ginkgo "--seed=1776792694" "--timeout=1h30m0s" "--grace-period=30s" "--output-interceptor-mode=none" "--no-color" "--json-report=e2e-report.json" "--junit-report=e2e-report.xml" "--procs=20" "--nodes=20" "--p" "--output-dir=/workspace/artifact-dir" "./cmd" "--" go: downloading github.com/konflux-ci/build-service v0.0.0-20240611083846-2dee6cfe6fe4 go: downloading github.com/IBM/go-sdk-core/v5 v5.15.3 go: downloading github.com/IBM/vpc-go-sdk v0.48.0 go: downloading github.com/aws/aws-sdk-go-v2 v1.41.3 go: downloading github.com/aws/aws-sdk-go-v2/config v1.31.3 go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.143.0 go: downloading github.com/minio/minio-go/v7 v7.0.99 go: downloading github.com/go-playground/validator/v10 v10.17.0 go: downloading github.com/go-openapi/strfmt v0.23.0 go: downloading github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 go: downloading github.com/google/go-github/v45 v45.2.0 go: downloading github.com/go-openapi/errors v0.22.2 go: downloading github.com/oklog/ulid v1.3.1 go: downloading go.mongodb.org/mongo-driver v1.17.4 go: downloading github.com/mitchellh/mapstructure v1.5.1-0.20231216201459-8508981c8b6c go: downloading github.com/leodido/go-urn v1.3.0 go: downloading github.com/gabriel-vasile/mimetype v1.4.3 go: downloading github.com/go-playground/universal-translator v0.18.1 go: downloading github.com/tinylib/msgp v1.6.1 go: downloading github.com/dustin/go-humanize v1.0.1 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/rs/xid v1.6.0 go: downloading github.com/klauspost/crc32 v1.3.0 go: downloading github.com/minio/crc64nvme v1.1.1 go: downloading github.com/minio/md5-simd v1.1.2 go: downloading github.com/aws/smithy-go v1.24.2 go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.28.2 go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.18.7 go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.4 go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.34.0 go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.38.0 go: downloading github.com/go-playground/locales v0.14.1 go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.19 go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.19 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.19 go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.6 go: downloading github.com/philhofer/fwd v1.2.0 Running Suite: Red Hat App Studio E2E tests - /tmp/tmp.dle8pz85Yc/cmd ===================================================================== Random Seed: 1776792694 Will run 375 of 463 specs Running in parallel across 20 processes ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rhtap-service-push pipeline] Rhtap-service-push happy path Post-release verification verifies if the release CR is created [release-pipelines, rhtap-service-push, RhtapServicePush] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rhtap_service_push.go:150 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for release-to-github pipeline] Release-to-github happy path Post-release verification verifies if release CR is created [release-pipelines, release-to-github, releaseToGithub] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/release_to_github.go:138 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rhtap-service-push pipeline] Rhtap-service-push happy path Post-release verification verifies the rhtap release pipelinerun is running and succeeds [release-pipelines, rhtap-service-push, RhtapServicePush] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rhtap_service_push.go:160 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for release-to-github pipeline] Release-to-github happy path Post-release verification verifies the release pipelinerun is running and succeeds [release-pipelines, release-to-github, releaseToGithub] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/release_to_github.go:148 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for release-to-github pipeline] Release-to-github happy path Post-release verification verifies release CR completed and set succeeded. [release-pipelines, release-to-github, releaseToGithub] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/release_to_github.go:181 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rhtap-service-push pipeline] Rhtap-service-push happy path Post-release verification verifies release CR completed and set succeeded. [release-pipelines, rhtap-service-push, RhtapServicePush] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rhtap_service_push.go:190 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for release-to-github pipeline] Release-to-github happy path Post-release verification verifies if the Release exists in github repo [release-pipelines, release-to-github, releaseToGithub] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/release_to_github.go:192 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rhtap-service-push pipeline] Rhtap-service-push happy path Post-release verification verifies if the PR in infra-deployments repo is created/updated [release-pipelines, rhtap-service-push, RhtapServicePush] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rhtap_service_push.go:200 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC happy path Post-release verification creates component from git source https://github.com/redhat-appstudio-qe/fbc-sample-repo-test [release-pipelines, fbc-release, fbcHappyPath] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:123 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC happy path Post-release verification Creates a push snapshot for a release [release-pipelines, fbc-release, fbcHappyPath] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:128 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC happy path Post-release verification verifies the fbc release pipelinerun is running and succeeds [release-pipelines, fbc-release, fbcHappyPath] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:132 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC happy path Post-release verification verifies release CR completed and set succeeded. [release-pipelines, fbc-release, fbcHappyPath] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:136 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC Staged Index Post-release verification creates component from git source https://github.com/redhat-appstudio-qe/fbc-sample-repo-test [release-pipelines, fbc-release, fbcStagedIndex] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:178 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC Staged Index Post-release verification Creates a push snapshot for a release [release-pipelines, fbc-release, fbcStagedIndex] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:183 ------------------------------ P [PENDING] [task-suite tkn bundle task] creates Tekton bundles with different params when context points to a file [build-templates] /tmp/tmp.dle8pz85Yc/tests/build/tkn-bundle.go:188 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC Staged Index Post-release verification verifies the fbc release pipelinerun is running and succeeds [release-pipelines, fbc-release, fbcStagedIndex] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:187 ------------------------------ P [PENDING] [task-suite tkn bundle task] creates Tekton bundles with different params creates Tekton bundles from specific context [build-templates] /tmp/tmp.dle8pz85Yc/tests/build/tkn-bundle.go:199 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC Staged Index Post-release verification verifies release CR completed and set succeeded. [release-pipelines, fbc-release, fbcStagedIndex] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:191 ------------------------------ P [PENDING] [task-suite tkn bundle task] creates Tekton bundles with different params when context is the root directory [build-templates] /tmp/tmp.dle8pz85Yc/tests/build/tkn-bundle.go:209 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC hotfix process FBC hotfix post-release verification creates component from git source https://github.com/redhat-appstudio-qe/fbc-sample-repo-test [release-pipelines, fbc-release, fbcHotfix] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:233 ------------------------------ P [PENDING] [task-suite tkn bundle task] creates Tekton bundles with different params creates Tekton bundles when context points to a file and a directory [build-templates] /tmp/tmp.dle8pz85Yc/tests/build/tkn-bundle.go:218 ------------------------------ P [PENDING] [task-suite tkn bundle task] creates Tekton bundles with different params creates Tekton bundles when using negation [build-templates] /tmp/tmp.dle8pz85Yc/tests/build/tkn-bundle.go:228 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC hotfix process FBC hotfix post-release verification Creates a push snapshot for a release [release-pipelines, fbc-release, fbcHotfix] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:238 ------------------------------ P [PENDING] [task-suite tkn bundle task] creates Tekton bundles with different params allows overriding HOME environment variable [build-templates] /tmp/tmp.dle8pz85Yc/tests/build/tkn-bundle.go:238 ------------------------------ P [PENDING] [task-suite tkn bundle task] creates Tekton bundles with different params allows overriding STEP image [build-templates] /tmp/tmp.dle8pz85Yc/tests/build/tkn-bundle.go:247 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC hotfix process FBC hotfix post-release verification verifies the fbc release pipelinerun is running and succeeds [release-pipelines, fbc-release, fbcHotfix] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:242 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for multi arch with rh-advisories pipeline] Multi arch test happy path Post-release verification verifies the release CR is created [release-pipelines, rh-advisories, multiarch-advisories, multiArchAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/multiarch_advisories.go:112 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC hotfix process FBC hotfix post-release verification verifies release CR completed and set succeeded. [release-pipelines, fbc-release, fbcHotfix] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:246 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-push-to-redhat-io pipeline] Rh-push-to-redhat-io happy path Post-release verification verifies if the release CR is created [release-pipelines, rh-push-to-registry-redhat-io, PushToRedhatIO] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_registry_redhat_io.go:108 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC pre-GA process FBC pre-GA post-release verification creates component from git source https://github.com/redhat-appstudio-qe/fbc-sample-repo-test [release-pipelines, fbc-release, fbcPreGA] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:286 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-push-to-redhat-io pipeline] Rh-push-to-redhat-io happy path Post-release verification verifies the rhio release pipelinerun is running and succeeds [release-pipelines, rh-push-to-registry-redhat-io, PushToRedhatIO] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_registry_redhat_io.go:118 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC pre-GA process FBC pre-GA post-release verification Creates a push snapshot for a release [release-pipelines, fbc-release, fbcPreGA] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:291 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for multi arch with rh-advisories pipeline] Multi arch test happy path Post-release verification verifies the multiarch release pipelinerun is running and succeeds [release-pipelines, rh-advisories, multiarch-advisories, multiArchAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/multiarch_advisories.go:122 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC pre-GA process FBC pre-GA post-release verification verifies the fbc release pipelinerun is running and succeeds [release-pipelines, fbc-release, fbcPreGA] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:295 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-push-to-redhat-io pipeline] Rh-push-to-redhat-io happy path Post-release verification verifies release CR completed and set succeeded. [release-pipelines, rh-push-to-registry-redhat-io, PushToRedhatIO] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_registry_redhat_io.go:148 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for multi arch with rh-advisories pipeline] Multi arch test happy path Post-release verification verifies release CR completed and set succeeded. [release-pipelines, rh-advisories, multiarch-advisories, multiArchAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/multiarch_advisories.go:152 ------------------------------ P [PENDING] [release-pipelines-suite FBC e2e-tests] with FBC pre-GA process FBC pre-GA post-release verification verifies release CR completed and set succeeded. [release-pipelines, fbc-release, fbcPreGA] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/fbc_release.go:299 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for multi arch with rh-advisories pipeline] Multi arch test happy path Post-release verification verifies if the repository URL is valid [release-pipelines, rh-advisories, multiarch-advisories, multiArchAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/multiarch_advisories.go:163 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-push-to-redhat-io pipeline] Rh-push-to-redhat-io happy path Post-release verification verifies if the MR URL is valid [release-pipelines, rh-push-to-registry-redhat-io, PushToRedhatIO] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_registry_redhat_io.go:159 ------------------------------ P [PENDING] [release-pipelines-suite [HACBS-1571]test-release-e2e-push-image-to-pyxis] Post-release verification tests that Release CR is created for the Snapshot [release-pipelines, rh-push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_external_registry.go:215 ------------------------------ P [PENDING] [release-pipelines-suite [HACBS-1571]test-release-e2e-push-image-to-pyxis] Post-release verification verifies a release PipelineRun is started and succeeded in managed namespace [release-pipelines, rh-push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_external_registry.go:226 ------------------------------ P [PENDING] [release-pipelines-suite [HACBS-1571]test-release-e2e-push-image-to-pyxis] Post-release verification validate the result of task create-pyxis-image contains image ids [release-pipelines, rh-push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_external_registry.go:233 ------------------------------ P [PENDING] [release-pipelines-suite [HACBS-1571]test-release-e2e-push-image-to-pyxis] Post-release verification tests that Release CR has completed [release-pipelines, rh-push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_external_registry.go:248 ------------------------------ P [PENDING] [release-pipelines-suite [HACBS-1571]test-release-e2e-push-image-to-pyxis] Post-release verification validates that imageIds from task create-pyxis-image exist in Pyxis. [release-pipelines, rh-push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_push_to_external_registry.go:265 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-advisories pipeline] Rh-advisories happy path Post-release verification verifies if release CR is created [release-pipelines, rh-advisories, rhAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_advisories.go:117 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-advisories pipeline] Rh-advisories happy path Post-release verification verifies the advs release pipelinerun is running and succeeds [release-pipelines, rh-advisories, rhAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_advisories.go:127 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-advisories pipeline] Rh-advisories happy path Post-release verification verifies release CR completed and set succeeded. [release-pipelines, rh-advisories, rhAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_advisories.go:157 ------------------------------ P [PENDING] [release-pipelines-suite e2e tests for rh-advisories pipeline] Rh-advisories happy path Post-release verification verifies if the repository URL is valid [release-pipelines, rh-advisories, rhAdvisories] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/rh_advisories.go:168 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws host-pool allocation when the Component with multi-platform-build is created a PipelineRun is triggered [multi-platform, aws-host-pool] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:120 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws host-pool allocation when the Component with multi-platform-build is created the build-container task from component pipelinerun is buildah-remote [multi-platform, aws-host-pool] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:124 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws host-pool allocation when the Component with multi-platform-build is created The multi platform secret is populated [multi-platform, aws-host-pool] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:127 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws host-pool allocation when the Component with multi-platform-build is created that PipelineRun completes successfully [multi-platform, aws-host-pool] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:148 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws host-pool allocation when the Component with multi-platform-build is created test that cleanup happened successfully [multi-platform, aws-host-pool] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:152 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws dynamic allocation when the Component with multi-platform-build is created a PipelineRun is triggered [multi-platform, aws-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:251 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws dynamic allocation when the Component with multi-platform-build is created the build-container task from component pipelinerun is buildah-remote [multi-platform, aws-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:255 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws dynamic allocation when the Component with multi-platform-build is created The multi platform secret is populated [multi-platform, aws-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:259 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws dynamic allocation when the Component with multi-platform-build is created that PipelineRun completes successfully [multi-platform, aws-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:263 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] aws dynamic allocation when the Component with multi-platform-build is created check cleanup happened successfully [multi-platform, aws-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:267 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm system z dynamic allocation when the Component with multi-platform-build is created a PipelineRun is triggered [multi-platform, ibmz-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:341 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm system z dynamic allocation when the Component with multi-platform-build is created the build-container task from component pipelinerun is buildah-remote [multi-platform, ibmz-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:345 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm system z dynamic allocation when the Component with multi-platform-build is created The multi platform secret is populated [multi-platform, ibmz-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:349 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm system z dynamic allocation when the Component with multi-platform-build is created that PipelineRun completes successfully [multi-platform, ibmz-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:353 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm system z dynamic allocation when the Component with multi-platform-build is created check cleanup happened successfully [multi-platform, ibmz-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:357 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm power pc dynamic allocation when the Component with multi-platform-build is created a PipelineRun is triggered [multi-platform, ibmp-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:432 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm power pc dynamic allocation when the Component with multi-platform-build is created the build-container task from component pipelinerun is buildah-remote [multi-platform, ibmp-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:436 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm power pc dynamic allocation when the Component with multi-platform-build is created The multi platform secret is populated [multi-platform, ibmp-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:440 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm power pc dynamic allocation when the Component with multi-platform-build is created that PipelineRun completes successfully [multi-platform, ibmp-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:444 ------------------------------ P [PENDING] [multi-platform-build-service-suite Multi Platform Controller E2E tests] ibm power pc dynamic allocation when the Component with multi-platform-build is created check cleanup happened successfully [multi-platform, ibmp-dynamic] /tmp/tmp.dle8pz85Yc/tests/build/multi-platform.go:448 ------------------------------ • [PANICKED] [39.437 seconds] [upgrade-suite Create users and check their state] [It] Verify AppStudioProvisionedUser [upgrade-verify] /tmp/tmp.dle8pz85Yc/tests/upgrade/verifyWorkload.go:20 Timeline >> "msg"="Observed a panic" "error"=null "panic"="runtime error: invalid memory address or nil pointer dereference" "panicGoValue"="\"invalid memory address or nil pointer dereference\"" "stacktrace"="goroutine 185 [running]:\nk8s.io/apimachinery/pkg/util/runtime.logPanic({0x49970d0, 0xc000de1da0}, {0x3861380, 0x6e91880})\n\t/opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:132 +0xbc\nk8s.io/apimachinery/pkg/util/runtime.handleCrash({0x4997178, 0xc000a30150}, {0x3861380, 0x6e91880}, {0x0, 0x0, 0x4470a0?})\n\t/opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:107 +0x116\nk8s.io/apimachinery/pkg/util/runtime.HandleCrashWithContext({0x4997178, 0xc000a30150}, {0x0, 0x0, 0x0})\n\t/opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:78 +0x5a\npanic({0x3861380?, 0x6e91880?})\n\t/usr/lib/golang/src/runtime/panic.go:783 +0x132\ngithub.com/konflux-ci/e2e-tests/pkg/sandbox.(*SandboxController).CheckUserCreatedWithSignUp.func1()\n\t/tmp/tmp.dle8pz85Yc/pkg/sandbox/sandbox.go:319 +0x35\ngithub.com/konflux-ci/e2e-tests/pkg/utils.WaitUntilWithInterval.func1({0xc000a30201?, 0xc0016a3c18?})\n\t/tmp/tmp.dle8pz85Yc/pkg/utils/util.go:129 +0x13\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func1({0x4997178?, 0xc000a30150?}, 0x4997178?)\n\t/opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/wait/loop.go:53 +0x62\nk8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext({0x4997178, 0xc000a30150}, {0x49811c8, 0xc000aae040}, 0x1, 0x0, 0xc0016a3d90)\n\t/opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/wait/loop.go:54 +0x115\nk8s.io/apimachinery/pkg/util/wait.PollUntilContextTimeout({0x4997028?, 0x6f54260?}, 0xee6b2800, 0x41d765?, 0x1, 0xc0016a3d90)\n\t/opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/wait/poll.go:48 +0xa5\ngithub.com/konflux-ci/e2e-tests/pkg/utils.WaitUntilWithInterval(0xa?, 0xc0016a3dd8?, 0x1?)\n\t/tmp/tmp.dle8pz85Yc/pkg/utils/util.go:129 +0x45\ngithub.com/konflux-ci/e2e-tests/pkg/sandbox.(*SandboxController).CheckUserCreatedWithSignUp(0x4005ab6?, {0x4005ab6?, 0x4002074?}, 0x8?)\n\t/tmp/tmp.dle8pz85Yc/pkg/sandbox/sandbox.go:318 +0x72\ngithub.com/konflux-ci/e2e-tests/pkg/sandbox.(*SandboxController).CheckUserCreated(0x0, {0x4005ab6, 0x9})\n\t/tmp/tmp.dle8pz85Yc/pkg/sandbox/sandbox.go:314 +0x4b\ngithub.com/konflux-ci/e2e-tests/tests/upgrade/verify.VerifyAppStudioProvisionedUser(0xc000a686b0?)\n\t/tmp/tmp.dle8pz85Yc/tests/upgrade/verify/verifyUsers.go:14 +0x25\ngithub.com/konflux-ci/e2e-tests/tests/upgrade.init.func1.2()\n\t/tmp/tmp.dle8pz85Yc/tests/upgrade/verifyWorkload.go:21 +0x1a\ngithub.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0xc000a68720?, 0xc3dbb5?})\n\t/opt/app-root/src/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.27.4/internal/node.go:524 +0x13\ngithub.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()\n\t/opt/app-root/src/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.27.4/internal/suite.go:945 +0x393\ncreated by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 100\n\t/opt/app-root/src/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.27.4/internal/suite.go:910 +0xd50\n" [PANICKED] in [It] - /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:114 @ 04/21/26 17:49:49.556 << Timeline [PANICKED] Test Panicked In [It] at: /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:114 @ 04/21/26 17:49:49.556 runtime error: invalid memory address or nil pointer dereference Full Stack Trace k8s.io/apimachinery/pkg/util/runtime.handleCrash({0x4997178, 0xc000a30150}, {0x3861380, 0x6e91880}, {0x0, 0x0, 0x4470a0?}) /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:114 +0x1a9 k8s.io/apimachinery/pkg/util/runtime.HandleCrashWithContext({0x4997178, 0xc000a30150}, {0x0, 0x0, 0x0}) /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:78 +0x5a panic({0x3861380?, 0x6e91880?}) /usr/lib/golang/src/runtime/panic.go:783 +0x132 github.com/konflux-ci/e2e-tests/pkg/sandbox.(*SandboxController).CheckUserCreatedWithSignUp.func1() /tmp/tmp.dle8pz85Yc/pkg/sandbox/sandbox.go:319 +0x35 github.com/konflux-ci/e2e-tests/pkg/utils.WaitUntilWithInterval.func1({0xc000a30201?, 0xc0016a3c18?}) /tmp/tmp.dle8pz85Yc/pkg/utils/util.go:129 +0x13 k8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext.func1({0x4997178?, 0xc000a30150?}, 0x4997178?) /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/wait/loop.go:53 +0x62 k8s.io/apimachinery/pkg/util/wait.loopConditionUntilContext({0x4997178, 0xc000a30150}, {0x49811c8, 0xc000aae040}, 0x1, 0x0, 0xc001521d90) /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/wait/loop.go:54 +0x115 k8s.io/apimachinery/pkg/util/wait.PollUntilContextTimeout({0x4997028?, 0x6f54260?}, 0xee6b2800, 0x41d765?, 0x1, 0xc0016a3d90) /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/wait/poll.go:48 +0xa5 github.com/konflux-ci/e2e-tests/pkg/utils.WaitUntilWithInterval(0xa?, 0xc0016a3dd8?, 0x1?) /tmp/tmp.dle8pz85Yc/pkg/utils/util.go:129 +0x45 github.com/konflux-ci/e2e-tests/pkg/sandbox.(*SandboxController).CheckUserCreatedWithSignUp(0x4005ab6?, {0x4005ab6?, 0x4002074?}, 0x8?) /tmp/tmp.dle8pz85Yc/pkg/sandbox/sandbox.go:318 +0x72 github.com/konflux-ci/e2e-tests/pkg/sandbox.(*SandboxController).CheckUserCreated(0x0, {0x4005ab6, 0x9}) /tmp/tmp.dle8pz85Yc/pkg/sandbox/sandbox.go:314 +0x4b github.com/konflux-ci/e2e-tests/tests/upgrade/verify.VerifyAppStudioProvisionedUser(0xc000a686b0?) /tmp/tmp.dle8pz85Yc/tests/upgrade/verify/verifyUsers.go:14 +0x25 github.com/konflux-ci/e2e-tests/tests/upgrade.init.func1.2() /tmp/tmp.dle8pz85Yc/tests/upgrade/verifyWorkload.go:21 +0x1a ------------------------------ SS•••••••••••••••••••••••••••••••••••••• ------------------------------ P [PENDING] [release-pipelines-suite Push to external registry] Post-release verification verifies that a Release CR should have been created in the dev namespace [release-pipelines, push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/push_to_external_registry.go:157 ------------------------------ P [PENDING] [release-pipelines-suite Push to external registry] Post-release verification verifies that Release PipelineRun should eventually succeed [release-pipelines, push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/push_to_external_registry.go:164 ------------------------------ P [PENDING] [release-pipelines-suite Push to external registry] Post-release verification tests if the image was pushed to quay [release-pipelines, push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/push_to_external_registry.go:168 ------------------------------ P [PENDING] [release-pipelines-suite Push to external registry] Post-release verification verifies that a Release is marked as succeeded. [release-pipelines, push-to-external-registry] /tmp/tmp.dle8pz85Yc/tests/release/pipelines/push_to_external_registry.go:175 ------------------------------ ••••••••••••••••••••••••••••••••••••••••••••••••••••••• ------------------------------ P [PENDING] [build-service-suite Build service E2E tests] test build secret lookup when two secrets are created when second component is deleted, pac pr branch should not exist in the repo [build-service, github, pac-build, secret-lookup] /tmp/tmp.dle8pz85Yc/tests/build/secret_lookup.go:206 ------------------------------ •••• ------------------------------ • [FAILED] [278.729 seconds] [enterprise-contract-suite Conforma E2E tests] test creating and signing an image and task verify-enterprise-contract task [It] succeeds when policy is met [ec, pipeline] /tmp/tmp.dle8pz85Yc/tests/enterprise-contract/contract.go:177 Timeline >> Copy public key from openshift-pipelines/signing-secrets to a new secret Configured Rekor host: https://rekor.sigstore.dev Using verify EC task bundle: quay.io/conforma/tekton-task:kf-cdfd9188f9352d7269ae1fe8c273a9e67f60ab8a@sha256:3801e78906a70b6dcf850a063cd90b8f5a785b3b54e39ba04630268b08703048 Creating Pipeline "verify-enterprise-contract-run-9mhs4" Waiting for pipeline "verify-enterprise-contract-run-9mhs4" to finish *** TaskRun status: artifacts: {} completionTime: "2026-04-21T17:58:32Z" conditions: - lastTransitionTime: "2026-04-21T17:58:32Z" message: '"step-show-config" exited with code 2: Error' reason: Failed status: "False" type: Succeeded podName: verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod provenance: featureFlags: awaitSidecarReadiness: true coschedule: workspaces enableAPIFields: alpha enableParamEnum: true enableProvenanceInStatus: true enforceNonfalsifiability: none maxResultSize: 4096 resultExtractionMethod: termination-message runningInEnvWithInjectedSidecars: true verificationNoMatchPolicy: ignore refSource: digest: sha256: 3801e78906a70b6dcf850a063cd90b8f5a785b3b54e39ba04630268b08703048 entryPoint: verify-enterprise-contract uri: quay.io/conforma/tekton-task spanContext: traceparent: 00-767a0abb6283cba2c77718dcd8a3443b-8541c8385f6fc839-01 startTime: "2026-04-21T17:53:56Z" steps: - container: step-initialize-tuf imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: initialize-tuf provenance: {} terminated: containerID: cri-o://3aae00289487dc93eeadec615ffe69368e234109841909884de8d83cc66a22a0 exitCode: 0 finishedAt: "2026-04-21T17:58:24Z" reason: Completed startedAt: "2026-04-21T17:58:24Z" terminationReason: Skipped - container: step-reduce imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: reduce provenance: {} terminated: containerID: cri-o://b37c913defed5d9b9c225b3b9fd34782c869931bf525ccd7e9a9e510373766b0 exitCode: 0 finishedAt: "2026-04-21T17:58:24Z" reason: Completed startedAt: "2026-04-21T17:58:24Z" terminationReason: Completed - container: step-validate imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: validate provenance: {} terminated: containerID: cri-o://3eeec84777e8b9c8faef53d8c83c8ef034ed6271c114318fb946e65650e25a39 exitCode: 1 finishedAt: "2026-04-21T17:58:29Z" reason: Completed startedAt: "2026-04-21T17:58:24Z" terminationReason: Continued - container: step-report-json imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: report-json provenance: {} terminated: containerID: cri-o://ea6dc769a7bb41a0415b218356847390c1c6468b3f8b0caa59bc20d2b7b74dee exitCode: 0 finishedAt: "2026-04-21T17:58:30Z" reason: Completed startedAt: "2026-04-21T17:58:29Z" terminationReason: Completed - container: step-summary imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: summary provenance: {} terminated: containerID: cri-o://5f68a09ea00eb2b58debd83a16f5b311810eb7ec73dc1d812ff2585cfdec8d9b exitCode: 2 finishedAt: "2026-04-21T17:58:30Z" reason: Completed startedAt: "2026-04-21T17:58:30Z" terminationReason: Continued - container: step-info imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: info provenance: {} terminated: containerID: cri-o://abc10a797c168d80761982293c8ef0d28aef32803ac0d1837086b76af4cf7787 exitCode: 0 finishedAt: "2026-04-21T17:58:32Z" reason: Completed startedAt: "2026-04-21T17:58:30Z" terminationReason: Completed - container: step-version imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: version provenance: {} terminated: containerID: cri-o://e534e70f51ecd170fdb6a44ecadb0f8c5d5ad05e3f146f919245efd3e63b0a72 exitCode: 0 finishedAt: "2026-04-21T17:58:32Z" reason: Completed startedAt: "2026-04-21T17:58:30Z" terminationReason: Completed - container: step-show-config imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: show-config provenance: {} terminated: containerID: cri-o://9a79bbb4eb1e161a06a0c3ae1d9ac930ce035aa69582f1223290ef6ab1522686 exitCode: 2 finishedAt: "2026-04-21T17:58:32Z" reason: Error startedAt: "2026-04-21T17:58:30Z" terminationReason: Error - container: step-assert imageID: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: assert provenance: {} terminated: containerID: cri-o://3280afb34bc2f8a8be544b57ca0668179a902733144df8d45a7223c01f4ee006 exitCode: 1 finishedAt: "2026-04-21T17:58:32Z" reason: Error startedAt: "2026-04-21T17:58:30Z" terminationReason: Skipped taskSpec: description: Verify the enterprise contract is met params: - description: | Spec section of an ApplicationSnapshot resource. Not all fields of the resource are required. A minimal example: ```json { "components": [ { "containerImage": "quay.io/example/repo:latest" } ] } ``` Each `containerImage` in the `components` array is validated. name: IMAGES type: string - default: enterprise-contract-service/default description: | Name of the policy configuration (EnterpriseContractPolicy resource) to use. `namespace/name` or `name` syntax supported. If namespace is omitted the namespace where the task runs is used. You can also specify a policy configuration using a git url, e.g. `github.com/conforma/config//slsa3`. name: POLICY_CONFIGURATION type: string - default: "" description: Public key used to verify signatures. Must be a valid k8s cosign reference, e.g. k8s://my-space/my-secret where my-secret contains the expected cosign.pub attribute. name: PUBLIC_KEY type: string - default: "" description: Rekor host for transparency log lookups name: REKOR_HOST type: string - default: "false" description: Skip Rekor transparency log checks during validation. name: IGNORE_REKOR type: string - default: "" description: TUF mirror URL. Provide a value when NOT using public sigstore deployment. name: TUF_MIRROR type: string - default: "" description: | Path to a directory containing SSL certs to be used when communicating with external services. This is useful when using the integrated registry and a local instance of Rekor on a development cluster which may use certificates issued by a not-commonly trusted root CA. In such cases, `/var/run/secrets/kubernetes.io/serviceaccount` is a good value. Multiple paths can be provided by using the `:` separator. name: SSL_CERT_DIR type: string - default: trusted-ca description: The name of the ConfigMap to read CA bundle data from. name: CA_TRUST_CONFIGMAP_NAME type: string - default: ca-bundle.crt description: The name of the key in the ConfigMap that contains the CA bundle data. name: CA_TRUST_CONFIG_MAP_KEY type: string - default: "true" description: Include rule titles and descriptions in the output. Set to `"false"` to disable it. name: INFO type: string - default: "true" description: Fail the task if policy fails. Set to `"false"` to disable it. name: STRICT type: string - default: /tekton/home description: Value for the HOME environment variable. name: HOMEDIR type: string - default: now description: Run policy checks with the provided time. name: EFFECTIVE_TIME type: string - default: "" description: Merge additional Rego variables into the policy data. Use syntax "key=value,key2=value2..." name: EXTRA_RULE_DATA type: string - default: "" description: | This param is deprecated and will be removed in future. Its value is ignored. EC will be run without a timeout. (If you do want to apply a timeout use the Tekton task timeout.) name: TIMEOUT type: string - default: "1" description: Number of parallel workers to use for policy evaluation. name: WORKERS type: string - default: "false" description: Reduce the Snapshot to only the component whose build caused the Snapshot to be created name: SINGLE_COMPONENT type: string - default: unknown description: | Name, including kind, of the Kubernetes resource to query for labels when single component mode is enabled, e.g. pr/somepipeline. name: SINGLE_COMPONENT_CUSTOM_RESOURCE type: string - default: "" description: | Kubernetes namespace where the SINGLE_COMPONENT_NAME is found. Only used when single component mode is enabled. name: SINGLE_COMPONENT_CUSTOM_RESOURCE_NS type: string results: - description: Short summary of the policy evaluation for each image name: TEST_OUTPUT type: string stepTemplate: computeResources: {} env: - name: HOME value: /tekton/home steps: - args: - sigstore - initialize - --mirror - "" - --root - /root.json command: - ec computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: initialize-tuf when: - operator: notin values: - "" - command: - reduce-snapshot.sh computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi env: - name: SNAPSHOT value: '{"application":"","components":[{"name":"","containerImage":"quay.io/redhat-appstudio-qe/test-images:buildah-demo-oubfrbzckr@sha256:f83d694a5619bf6fbdd50d98c682862ab11a3f09491dd672e69982fd5ffec2b3","source":{}}],"artifacts":{}}' - name: SINGLE_COMPONENT value: "false" - name: CUSTOM_RESOURCE value: unknown - name: CUSTOM_RESOURCE_NAMESPACE - name: SNAPSHOT_PATH value: /tekton/home/snapshot.json image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: reduce onError: continue - args: - validate - image - --images - /tekton/home/snapshot.json - --policy - ec-policy - --public-key - k8s://chains-e2e-sary/cosign-public-key - --rekor-url - "" - --ignore-rekor=true - --workers - "1" - --info=true - --timeout=100h - --strict=false - --show-successes - --effective-time=now - --extra-rule-data= - --output - text?show-successes=false - --output - appstudio=/tekton/results/TEST_OUTPUT - --output - json=/tekton/home/report-json.json command: - ec computeResources: limits: memory: 2Gi requests: cpu: 1800m memory: 2Gi env: - name: SSL_CERT_DIR value: /tekton-custom-certs:/etc/ssl/certs:/etc/pki/tls/certs:/system/etc/security/cacerts:/var/run/secrets/kubernetes.io/serviceaccount - name: EC_CACHE value: "false" image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: validate onError: continue volumeMounts: - mountPath: /etc/pki/tls/certs/ca-custom-bundle.crt name: trusted-ca readOnly: true subPath: ca-bundle.crt - args: - jq . /tekton/home/report-json.json | awk '{gsub(/^ +/, ""); acc += length; if (acc >= 8000) { printf "\n"; acc=length } printf $0 }' command: - sh - -c computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: report-json onError: continue - args: - . - /tekton/results/TEST_OUTPUT command: - jq computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: summary onError: continue - args: - | ----- DEBUG OUTPUT ----- command: - printf computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: info - args: - version command: - ec computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: version - args: - '{policy: .policy, key: .key, "effective-time": .["effective-time"]}' - /tekton/home/report-json.json command: - jq computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: show-config - args: - --argjson - strict - "true" - -e - | .result == "SUCCESS" or .result == "WARNING" or ($strict | not) - /tekton/results/TEST_OUTPUT command: - jq computeResources: limits: memory: 256Mi requests: cpu: 100m memory: 256Mi image: quay.io/conforma/cli@sha256:9275ec5f062399135ecb3f54f520ee5f20e818a846c34250e0d417dce9221459 name: assert volumes: - configMap: items: - key: ca-bundle.crt path: ca-bundle.crt name: trusted-ca optional: true name: trusted-ca workspaces: - description: The workspace where the snapshot spec json file resides name: data optional: true *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-initialize-tuf': ----- START -----2026/04/21 17:58:24 INFO Step was skipped due to when expressions were evaluated to false. ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-reduce': ----- START -----Single Component mode? false { "application": "", "components": [ { "name": "", "containerImage": "quay.io/redhat-appstudio-qe/test-images:buildah-demo-oubfrbzckr@sha256:f83d694a5619bf6fbdd50d98c682862ab11a3f09491dd672e69982fd5ffec2b3", "source": {} } ], "artifacts": {} } ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-validate': ----- START -----Error: error validating image quay.io/redhat-appstudio-qe/test-images:buildah-demo-oubfrbzckr@sha256:f83d694a5619bf6fbdd50d98c682862ab11a3f09491dd672e69982fd5ffec2b3 of component : load: loading policies: get compiler: 5 errors occurred: /tmp/ec-work-558957108/policy/ecc96998b/policy/lib/sbom/sbom.rego:88: rego_type_error: undefined function ec.oci.image_referrers /tmp/ec-work-558957108/policy/ecc96998b/policy/lib/sbom/sbom.rego:98: rego_type_error: undefined function ec.oci.image_tag_refs /tmp/ec-work-558957108/policy/ecc96998b/policy/release/slsa_build_scripted_build/slsa_build_scripted_build.rego:23: rego_type_error: undefined function ec.oci.image_manifests /tmp/ec-work-558957108/policy/ecc96998b/policy/release/tasks/tasks.rego:37: rego_type_error: undefined function ec.oci.image_manifests /tmp/ec-work-558957108/policy/ecc96998b/policy/release/trusted_task/trusted_task.rego:23: rego_type_error: undefined function ec.oci.image_manifests ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-report-json': ----- START -----jq: error: Could not open file /tekton/home/report-json.json: No such file or directory ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-summary': ----- START -----jq: error: Could not open file /tekton/results/TEST_OUTPUT: No such file or directory ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-info': ----- START ---------- DEBUG OUTPUT ----- ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-version': ----- START -----Version v0.7.111 Source ID cdfd9188f9352d7269ae1fe8c273a9e67f60ab8a Change date 2025-07-09 12:58:49 +0000 UTC (40 weeks ago) ECC v0.1.112 OPA v0.70.0 Conftest v0.55.0 Cosign v2.4.1 Sigstore v1.8.9 Rekor v1.3.6 Tekton Pipeline v0.63.0 Kubernetes Client v0.31.0 ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-show-config': ----- START -----jq: error: Could not open file /tekton/home/report-json.json: No such file or directory ----- END ----- *** Logs from pod 'verify-enterprise-contract-4900f56bd67ae8da4d7945808fc9ad40-pod', container 'step-assert': ----- START -----2026/04/21 17:58:30 Skipping step because a previous step failed ----- END ----- Make sure TaskRun verify-enterprise-contract of PipelineRun verify-enterprise-contract-run-9mhs4 succeeded [FAILED] in [It] - /tmp/tmp.dle8pz85Yc/tests/enterprise-contract/contract.go:190 @ 04/21/26 17:58:34.032 << Timeline [FAILED] Expected : false to be true In [It] at: /tmp/tmp.dle8pz85Yc/tests/enterprise-contract/contract.go:190 @ 04/21/26 17:58:34.032 ------------------------------ SSSSSSSS•••••• ------------------------------ • [FAILED] [561.275 seconds] [integration-service-suite Status Reporting of Integration tests] with status reporting of Integration tests in CheckRuns when a new Component with specified custom branch is created [It] should lead to build PipelineRun finishing successfully [integration-service, github-status-reporting, custom-branch] /tmp/tmp.dle8pz85Yc/tests/integration-service/status-reporting-to-pullrequest.go:142 Timeline >> PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Running PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: PipelineRunStopping PipelineRun test-component-pac-icyyis-on-pull-request-9bs7d reason: Failed [FAILED] in [It] - /tmp/tmp.dle8pz85Yc/tests/integration-service/status-reporting-to-pullrequest.go:144 @ 04/21/26 18:00:08.392 << Timeline [FAILED] build pipelinerun fails for NameSpace/Application/Component stat-rep-ujhi/integ-app-uszb/test-component-pac-icyyis with logs: Pipelinerun 'test-component-pac-icyyis-on-pull-request-9bs7d' didn't succeed Expected success, but got an error: <*errors.errorString | 0xc0007a48c0>: Pipelinerun 'test-component-pac-icyyis-on-pull-request-9bs7d' didn't succeed { s: "Pipelinerun 'test-component-pac-icyyis-on-pull-request-9bs7d' didn't succeed\n", } In [It] at: /tmp/tmp.dle8pz85Yc/tests/integration-service/status-reporting-to-pullrequest.go:144 @ 04/21/26 18:00:08.392 ------------------------------ SSSSSSSSSSSSSSSSSSS••SS•S• ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build-oci-ta-min) when Pipeline Results are stored for component with Git source URL https://github.com/redhat-appstudio-qe/devfile-sample-python-basic and Pipeline docker-build-oci-ta-min should have Pipeline Logs [build, build-templates, HACBS, pipeline-service, pipeline] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:507 ------------------------------ • ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build-oci-ta-min) when the container image for component with Git source URL https://github.com/redhat-appstudio-qe/devfile-sample-python-basic is created and pushed to container registry verify-enterprise-contract check should pass [build, build-templates, HACBS, pipeline-service, pipeline, sbom, slow, build-templates-e2e] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:569 ------------------------------ S ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build-oci-ta-min) build-definitions ec pipelines runs ec pipeline pipelines/enterprise-contract.yaml [build, build-templates, HACBS, pipeline-service, pipeline, build-templates-e2e] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:744 ------------------------------ ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••S•S• ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build) when Pipeline Results are stored for component with Git source URL https://github.com/redhat-appstudio-qe/devfile-sample-python-basic and Pipeline docker-build should have Pipeline Logs [build, build-templates, HACBS, pipeline-service, pipeline] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:507 ------------------------------ • ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build) when the container image for component with Git source URL https://github.com/redhat-appstudio-qe/devfile-sample-python-basic is created and pushed to container registry verify-enterprise-contract check should pass [build, build-templates, HACBS, pipeline-service, pipeline, sbom, slow, build-templates-e2e] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:569 ------------------------------ S ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build) build-definitions ec pipelines runs ec pipeline pipelines/enterprise-contract.yaml [build, build-templates, HACBS, pipeline-service, pipeline, build-templates-e2e] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:744 ------------------------------ ••••••••••••••••••••••••••••••••••S•S• ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build-oci-ta) when Pipeline Results are stored for component with Git source URL https://github.com/redhat-appstudio-qe/devfile-sample-python-basic and Pipeline docker-build-oci-ta should have Pipeline Logs [build, build-templates, HACBS, pipeline-service, pipeline] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:507 ------------------------------ • ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build-oci-ta) when the container image for component with Git source URL https://github.com/redhat-appstudio-qe/devfile-sample-python-basic is created and pushed to container registry verify-enterprise-contract check should pass [build, build-templates, HACBS, pipeline-service, pipeline, sbom, slow, build-templates-e2e] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:569 ------------------------------ S ------------------------------ P [PENDING] [build-service-suite Build templates E2E test] HACBS pipelines scenario sample-python-basic-oci (docker-build-oci-ta) build-definitions ec pipelines runs ec pipeline pipelines/enterprise-contract.yaml [build, build-templates, HACBS, pipeline-service, pipeline, build-templates-e2e] /tmp/tmp.dle8pz85Yc/tests/build/build_templates.go:744 ------------------------------ ••••••••••••••••••••••••••••••••• ------------------------------ P [PENDING] [build-service-suite Build service E2E tests] test git provider gl PaC component build when the PaC init branch is merged retrigger the pipeline manually [build-service, github-webhook, pac-build, pipeline, image-controller, gitlab, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:600 ------------------------------ P [PENDING] [build-service-suite Build service E2E tests] test git provider gl PaC component build when the PaC init branch is merged retriggered pipelineRun should eventually finish [build-service, github-webhook, pac-build, pipeline, image-controller, gitlab, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:642 ------------------------------ ••••••••••••• ------------------------------ P [PENDING] [build-service-suite Build service E2E tests] test git provider gh PaC component build when the PaC init branch is merged retrigger the pipeline manually [build-service, github-webhook, pac-build, pipeline, image-controller, github, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:600 ------------------------------ P [PENDING] [build-service-suite Build service E2E tests] test git provider gh PaC component build when the PaC init branch is merged retriggered pipelineRun should eventually finish [build-service, github-webhook, pac-build, pipeline, image-controller, github, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:642 ------------------------------ ••••••••••••• ------------------------------ P [PENDING] [build-service-suite Build service E2E tests] test git provider fj PaC component build when the PaC init branch is merged retrigger the pipeline manually [build-service, github-webhook, pac-build, pipeline, image-controller, forgejo, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:600 ------------------------------ P [PENDING] [build-service-suite Build service E2E tests] test git provider fj PaC component build when the PaC init branch is merged retriggered pipelineRun should eventually finish [build-service, github-webhook, pac-build, pipeline, image-controller, forgejo, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:642 ------------------------------ • [FAILED] [24.737 seconds] [build-service-suite Build service E2E tests] test git provider fj PaC component build when the component is removed and recreated (with the same name in the same namespace) [It] should no longer lead to a creation of a PaC PR [build-service, github-webhook, pac-build, pipeline, image-controller, forgejo, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:715 Timeline >> [FAILED] in [It] - /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:732 @ 04/21/26 18:22:01.117 Found purge PR with id: 4 << Timeline [FAILED] Failed after 10.202s. Unexpected error: <*errors.errorString | 0xc0014bac20>: did not expect a new PR created in konflux-qe/devfile-sample-hello-world-lxapfv repository after initial PaC configuration was already merged for the same component name and a namespace { s: "did not expect a new PR created in konflux-qe/devfile-sample-hello-world-lxapfv repository after initial PaC configuration was already merged for the same component name and a namespace", } occurred In [It] at: /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:732 @ 04/21/26 18:22:01.117 ------------------------------ ••••••••• ------------------------------ • [FAILED] [1219.358 seconds] [build-service-suite Build service E2E tests] test git provider fj component update with renovate when components are created in same namespace [It] should lead to a nudge PR creation for child component [build-service, renovate, multi-component, forgejo] /tmp/tmp.dle8pz85Yc/tests/build/renovate.go:412 [FAILED] Timed out after 1200.001s. timed out when waiting for component nudge PR to be created in build-nudge-child-pxaexf repository Expected : false to be true In [It] at: /tmp/tmp.dle8pz85Yc/tests/build/renovate.go:427 @ 04/21/26 18:30:01.088 ------------------------------ SS•••••• ------------------------------ • [FAILED] [1004.091 seconds] [build-service-suite Build service E2E tests] test pac with multiple components using same repository when components are created in same namespace [It] the PipelineRun should eventually finish successfully for component python-component-sjporj [build-service, github, pac-build, multi-component] /tmp/tmp.dle8pz85Yc/tests/build/multi_component.go:159 Timeline >> PipelineRun python-component-sjporj-on-pull-request-gbc58 found for Component build-e2e-uurm/python-component-sjporj PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Running PipelineRun python-component-sjporj-on-pull-request-gbc58 reason: Failed attempt 1/3: PipelineRun "python-component-sjporj-on-pull-request-gbc58" failed: pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | init container: prepare 2026/04/21 18:42:48 Entrypoint initialization pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | init container: place-scripts 2026/04/21 18:42:48 Decoded script /tekton/scripts/script-1-8ctvf 2026/04/21 18:42:48 Decoded script /tekton/scripts/script-2-vmmj5 2026/04/21 18:42:48 Decoded script /tekton/scripts/script-3-nxdgv 2026/04/21 18:42:48 Decoded script /tekton/scripts/script-4-mwpv4 2026/04/21 18:42:48 Decoded script /tekton/scripts/script-5-x7sz6 pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | container step-use-trusted-artifact: Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras blob fetch --registry-config /tmp/use-oci.sh.ofyTL7/auth-zYpMXG.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:72f2d6ba8dc660a875468b6514252077de32cd615a28fca2ab6b2d1ae148402a --output - Restored artifact quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:72f2d6ba8dc660a875468b6514252077de32cd615a28fca2ab6b2d1ae148402a to /var/workdir/source WARN: artifact URI not provided, (given: =/var/workdir/cachi2) pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | container step-build: [2026-04-21T18:42:52,973463563+00:00] Validate context path [2026-04-21T18:42:52,976825012+00:00] Update CA trust [2026-04-21T18:42:52,977921811+00:00] Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:42:57,163150992+00:00] Prepare Dockerfile Checking if /var/workdir/cachi2/output/bom.json exists. Could not find prefetched sbom. No content_sets found for ICM [2026-04-21T18:42:57,168785446+00:00] Prepare system (architecture: x86_64) [2026-04-21T18:42:57,180207798+00:00] Setup prefetched Trying to pull registry.access.redhat.com/ubi9/python-39:1-117.1684741281... Getting image source signatures Checking if image destination supports signatures Copying blob sha256:8a2e9815396eaa57b0dfe27b2dfc442417f06aa70974c358f198980b533c7942 Copying blob sha256:5465449a2ea5d8a7e05fa498c8ba35f0ce7714b0ad37e716961fca17dfa1ef13 Copying blob sha256:085efe85e9dfb0b3d5d2392e4b78660a3091bf1ded336315117a627fbdd0567d Copying blob sha256:a07d808ddf4404b3885fef6100142a973cc667b2c3c4abcf4db96508795b1efb Copying config sha256:45c8f23c11bb64f238236aaf5cbe05595778e1d4fb6c06109c6fcc0a6b57a45b Writing manifest to image destination Storing signatures [2026-04-21T18:43:36,034980408+00:00] Unsetting proxy { "architecture": "x86_64", "build-date": "2026-04-21T18:42:57Z", "com.redhat.component": "python-39-container", "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI", "description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "distribution-scope": "public", "io.buildah.version": "1.42.2", "io.buildpacks.stack.id": "com.redhat.stacks.ubi9-python-39", "io.k8s.description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "io.k8s.display-name": "Python 3.9", "io.openshift.expose-services": "8080:http", "io.openshift.s2i.scripts-url": "image:///usr/libexec/s2i", "io.openshift.tags": "builder,python,python39,python-39,rh-python39", "io.s2i.scripts-url": "image:///usr/libexec/s2i", "maintainer": "SoftwareCollections.org ", "name": "ubi9/python-39", "release": "117.1684741281", "summary": "Platform for building and running Python 3.9 applications", "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/python-39/images/1-117.1684741281", "usage": "s2i build https://github.com/sclorg/s2i-python-container.git --context-dir=3.9/test/setup-test-app/ ubi9/python-39 python-sample-app", "vcs-ref": "d810ebca6da86ea152447936fab0f99aaa00ae35", "vcs-type": "git", "vendor": "Red Hat, Inc.", "version": "1", "org.opencontainers.image.revision": "d810ebca6da86ea152447936fab0f99aaa00ae35", "org.opencontainers.image.source": "https://github.com/redhat-appstudio-qe/sample-multi-component", "quay.expires-after": "6h", "org.opencontainers.image.created": "2026-04-21T18:42:57Z" } [2026-04-21T18:43:36,145882585+00:00] Register sub-man Adding the entitlement to the build [2026-04-21T18:43:36,149037891+00:00] Add secrets [2026-04-21T18:43:36,156296801+00:00] Run buildah build [2026-04-21T18:43:36,157391936+00:00] buildah build --volume /tmp/entitlement:/etc/pki/entitlement --security-opt=unmask=/proc/interrupts --label architecture=x86_64 --label vcs-type=git --label vcs-ref=d810ebca6da86ea152447936fab0f99aaa00ae35 --label org.opencontainers.image.revision=d810ebca6da86ea152447936fab0f99aaa00ae35 --label org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --label quay.expires-after=6h --label build-date=2026-04-21T18:42:57Z --label org.opencontainers.image.created=2026-04-21T18:42:57Z --annotation org.opencontainers.image.revision=d810ebca6da86ea152447936fab0f99aaa00ae35 --annotation org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --annotation org.opencontainers.image.created=2026-04-21T18:42:57Z --tls-verify=true --no-cache --ulimit nofile=4096:4096 --http-proxy=false -f /tmp/Dockerfile.dOn5kF -t quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-d810ebca6da86ea152447936fab0f99aaa00ae35 . STEP 1/10: FROM registry.access.redhat.com/ubi9/python-39:1-117.1684741281 STEP 2/10: EXPOSE 8081/tcp STEP 3/10: ENV FLASK_PORT=8081 STEP 4/10: WORKDIR /projects STEP 5/10: COPY . . STEP 6/10: RUN if [ -f requirements.txt ]; then pip install -r requirements.txt; elif [ `ls -1q *.txt | wc -l` == 1 ]; then pip install -r *.txt; fi Collecting Flask==2.3.3 Downloading flask-2.3.3-py3-none-any.whl (96 kB) Collecting Werkzeug>=2.3.7 Downloading werkzeug-3.1.8-py3-none-any.whl (226 kB) Collecting blinker>=1.6.2 Downloading blinker-1.9.0-py3-none-any.whl (8.5 kB) Collecting Jinja2>=3.1.2 Downloading jinja2-3.1.6-py3-none-any.whl (134 kB) Collecting click>=8.1.3 Downloading click-8.1.8-py3-none-any.whl (98 kB) Collecting itsdangerous>=2.1.2 Downloading itsdangerous-2.2.0-py3-none-any.whl (16 kB) Collecting importlib-metadata>=3.6.0 Downloading importlib_metadata-8.7.1-py3-none-any.whl (27 kB) Collecting zipp>=3.20 Downloading zipp-3.23.1-py3-none-any.whl (10 kB) Collecting MarkupSafe>=2.0 Downloading markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB) Installing collected packages: zipp, MarkupSafe, Werkzeug, Jinja2, itsdangerous, importlib-metadata, click, blinker, Flask Successfully installed Flask-2.3.3 Jinja2-3.1.6 MarkupSafe-3.0.3 Werkzeug-3.1.8 blinker-1.9.0 click-8.1.8 importlib-metadata-8.7.1 itsdangerous-2.2.0 zipp-3.23.1 WARNING: You are using pip version 21.3.1; however, version 26.0.1 is available. You should consider upgrading via the '/opt/app-root/bin/python3.9 -m pip install --upgrade pip' command. STEP 7/10: CMD [ "python", "./app.py" ] STEP 8/10: COPY labels.json /usr/share/buildinfo/labels.json STEP 9/10: COPY labels.json /root/buildinfo/labels.json STEP 10/10: LABEL "architecture"="x86_64" "vcs-type"="git" "vcs-ref"="d810ebca6da86ea152447936fab0f99aaa00ae35" "org.opencontainers.image.revision"="d810ebca6da86ea152447936fab0f99aaa00ae35" "org.opencontainers.image.source"="https://github.com/redhat-appstudio-qe/sample-multi-component" "quay.expires-after"="6h" "build-date"="2026-04-21T18:42:57Z" "org.opencontainers.image.created"="2026-04-21T18:42:57Z" COMMIT quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-d810ebca6da86ea152447936fab0f99aaa00ae35 --> 17b11a9b5736 Successfully tagged quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-d810ebca6da86ea152447936fab0f99aaa00ae35 17b11a9b5736bed90007f3d7c9a52a9827f03f1db75267c8f53c40bd58f3c97a [2026-04-21T18:43:43,344796448+00:00] Unsetting proxy [2026-04-21T18:43:43,346055622+00:00] Add metadata Recording base image digests used registry.access.redhat.com/ubi9/python-39:1-117.1684741281 registry.access.redhat.com/ubi9/python-39:1-117.1684741281@sha256:40a58935b9c22664927b22bf256f53a3d744ddb7316f3af18061099e199526ee Getting image source signatures Copying blob sha256:2b06cbeff05c3c2757d71ebd351aa2c35ce6a47424f6bb031412741d469e460e Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying config sha256:17b11a9b5736bed90007f3d7c9a52a9827f03f1db75267c8f53c40bd58f3c97a Writing manifest to image destination [2026-04-21T18:44:30,627695766+00:00] End build pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | container step-push: [2026-04-21T18:44:31,110165769+00:00] Update CA trust INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:44:53,438449333+00:00] Convert image [2026-04-21T18:44:53,439510409+00:00] Push image with unique tag Pushing to quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-gbc58-build-container [retry] executing: buildah push --format=docker --retry 3 --tls-verify=true quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-d810ebca6da86ea152447936fab0f99aaa00ae35 docker://quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-gbc58-build-container Getting image source signatures Copying blob sha256:2b06cbeff05c3c2757d71ebd351aa2c35ce6a47424f6bb031412741d469e460e Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | container step-sbom-syft-generate: pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | container step-prepare-sboms: pod: python-component-sjporj-on-810ea7f547743f7eee70e24547f40035-pod | container step-upload-sbom: pod: python-component-sjporj-on-e7266af3a1ae2341f678481ddbf92d16-pod | init container: prepare 2026/04/21 18:41:56 Entrypoint initialization pod: python-component-sjporj-on-e7266af3a1ae2341f678481ddbf92d16-pod | init container: place-scripts 2026/04/21 18:41:57 Decoded script /tekton/scripts/script-0-lq4wp 2026/04/21 18:41:57 Decoded script /tekton/scripts/script-1-prcnm pod: python-component-sjporj-on-e7266af3a1ae2341f678481ddbf92d16-pod | container step-clone: INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt {"level":"info","ts":1776796921.290442,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776796921.5157535,"caller":"git/git.go:223","msg":"Successfully cloned https://github.com/redhat-appstudio-qe/sample-multi-component @ d810ebca6da86ea152447936fab0f99aaa00ae35 (grafted, HEAD) in path /var/workdir/source"} {"level":"info","ts":1776796921.515802,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776796921.5416613,"caller":"git/git.go:277","msg":"Successfully initialized and updated submodules in path /var/workdir/source"} Merge option disabled. Using checked-out revision d810ebca6da86ea152447936fab0f99aaa00ae35 directly. pod: python-component-sjporj-on-e7266af3a1ae2341f678481ddbf92d16-pod | container step-symlink-check: Running symlink check pod: python-component-sjporj-on-e7266af3a1ae2341f678481ddbf92d16-pod | container step-create-trusted-artifact: Prepared artifact from /var/workdir/source (sha256:72f2d6ba8dc660a875468b6514252077de32cd615a28fca2ab6b2d1ae148402a) Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras push --annotation=quay.expires-after=6h --registry-config /tmp/create-oci.sh.aNZPq8/auth-jrZBTA.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-d810ebca6da86ea152447936fab0f99aaa00ae35.git SOURCE_ARTIFACT Uploading 72f2d6ba8dc6 SOURCE_ARTIFACT Uploaded 72f2d6ba8dc6 SOURCE_ARTIFACT Pushed [registry] quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-d810ebca6da86ea152447936fab0f99aaa00ae35.git ArtifactType: application/vnd.unknown.artifact.v1 Digest: sha256:1860fcd00a49d5eb6f8013963b4575fcb79417deba31a7fb5643da414d2a5649 Artifacts created pod: python-component-sjporj-on-pull-request-gbc58-init-pod | init container: prepare 2026/04/21 18:41:51 Entrypoint initialization pod: python-component-sjporj-on-pull-request-gbc58-init-pod | container step-init: time="2026-04-21T18:41:54Z" level=info msg="Using in-cluster config" logger=KubeClient time="2026-04-21T18:41:54Z" level=info msg="[param] enable: false" time="2026-04-21T18:41:54Z" level=info msg="[param] default-http-proxy: squid.caching.svc.cluster.local:3128" time="2026-04-21T18:41:54Z" level=info msg="[param] default-no-proxy: brew.registry.redhat.io,docker.io,gcr.io,ghcr.io,images.paas.redhat.com,mirror.gcr.io,nvcr.io,quay.io,registry-proxy.engineering.redhat.com,registry.access.redhat.com,registry.ci.openshift.org,registry.fedoraproject.org,registry.redhat.io,registry.stage.redhat.io,vault.habana.ai" time="2026-04-21T18:41:54Z" level=info msg="[param] http-proxy-result-path: /tekton/results/http-proxy" time="2026-04-21T18:41:54Z" level=info msg="[param] no-proxy-result-path: /tekton/results/no-proxy" time="2026-04-21T18:41:54Z" level=info msg="Cache proxy is disabled in param or in backend" time="2026-04-21T18:41:54Z" level=info msg="[result] HTTP PROXY: " time="2026-04-21T18:41:54Z" level=info msg="[result] NO PROXY: " New PipelineRun python-component-sjporj-on-pull-request-62np9 found after retrigger for component build-e2e-uurm/python-component-sjporj PipelineRun python-component-sjporj-on-pull-request-62np9 found for Component build-e2e-uurm/python-component-sjporj PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Running PipelineRun python-component-sjporj-on-pull-request-62np9 reason: Failed attempt 2/3: PipelineRun "python-component-sjporj-on-pull-request-62np9" failed: pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | init container: prepare 2026/04/21 18:47:46 Entrypoint initialization pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | init container: place-scripts 2026/04/21 18:47:47 Decoded script /tekton/scripts/script-1-hzq6j 2026/04/21 18:47:47 Decoded script /tekton/scripts/script-2-q2sc9 2026/04/21 18:47:47 Decoded script /tekton/scripts/script-3-ndhjm 2026/04/21 18:47:47 Decoded script /tekton/scripts/script-4-vg68c 2026/04/21 18:47:47 Decoded script /tekton/scripts/script-5-6r6w6 pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | container step-use-trusted-artifact: Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras blob fetch --registry-config /tmp/use-oci.sh.S6TQbV/auth-9rQs5h.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:47e1aac606956ea676ecf6d0943ca92a00c8566946c44fb0b30f5cdbb27bea40 --output - Restored artifact quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:47e1aac606956ea676ecf6d0943ca92a00c8566946c44fb0b30f5cdbb27bea40 to /var/workdir/source WARN: artifact URI not provided, (given: =/var/workdir/cachi2) pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | container step-build: [2026-04-21T18:47:52,884748706+00:00] Validate context path [2026-04-21T18:47:52,888429564+00:00] Update CA trust [2026-04-21T18:47:52,889645965+00:00] Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:47:57,160205416+00:00] Prepare Dockerfile Checking if /var/workdir/cachi2/output/bom.json exists. Could not find prefetched sbom. No content_sets found for ICM [2026-04-21T18:47:57,165957062+00:00] Prepare system (architecture: x86_64) [2026-04-21T18:47:57,230135101+00:00] Setup prefetched Trying to pull registry.access.redhat.com/ubi9/python-39:1-117.1684741281... Getting image source signatures Checking if image destination supports signatures Copying blob sha256:8a2e9815396eaa57b0dfe27b2dfc442417f06aa70974c358f198980b533c7942 Copying blob sha256:5465449a2ea5d8a7e05fa498c8ba35f0ce7714b0ad37e716961fca17dfa1ef13 Copying blob sha256:085efe85e9dfb0b3d5d2392e4b78660a3091bf1ded336315117a627fbdd0567d Copying blob sha256:a07d808ddf4404b3885fef6100142a973cc667b2c3c4abcf4db96508795b1efb Copying config sha256:45c8f23c11bb64f238236aaf5cbe05595778e1d4fb6c06109c6fcc0a6b57a45b Writing manifest to image destination Storing signatures [2026-04-21T18:49:25,105393737+00:00] Unsetting proxy { "architecture": "x86_64", "build-date": "2026-04-21T18:47:57Z", "com.redhat.component": "python-39-container", "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI", "description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "distribution-scope": "public", "io.buildah.version": "1.42.2", "io.buildpacks.stack.id": "com.redhat.stacks.ubi9-python-39", "io.k8s.description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "io.k8s.display-name": "Python 3.9", "io.openshift.expose-services": "8080:http", "io.openshift.s2i.scripts-url": "image:///usr/libexec/s2i", "io.openshift.tags": "builder,python,python39,python-39,rh-python39", "io.s2i.scripts-url": "image:///usr/libexec/s2i", "maintainer": "SoftwareCollections.org ", "name": "ubi9/python-39", "release": "117.1684741281", "summary": "Platform for building and running Python 3.9 applications", "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/python-39/images/1-117.1684741281", "usage": "s2i build https://github.com/sclorg/s2i-python-container.git --context-dir=3.9/test/setup-test-app/ ubi9/python-39 python-sample-app", "vcs-ref": "0e5846e2d353f52487421db684a71b1865ef81ab", "vcs-type": "git", "vendor": "Red Hat, Inc.", "version": "1", "org.opencontainers.image.revision": "0e5846e2d353f52487421db684a71b1865ef81ab", "org.opencontainers.image.source": "https://github.com/redhat-appstudio-qe/sample-multi-component", "quay.expires-after": "6h", "org.opencontainers.image.created": "2026-04-21T18:47:57Z" } [2026-04-21T18:49:25,232215159+00:00] Register sub-man Adding the entitlement to the build [2026-04-21T18:49:25,235695086+00:00] Add secrets [2026-04-21T18:49:25,243385048+00:00] Run buildah build [2026-04-21T18:49:25,244533188+00:00] buildah build --volume /tmp/entitlement:/etc/pki/entitlement --security-opt=unmask=/proc/interrupts --label architecture=x86_64 --label vcs-type=git --label vcs-ref=0e5846e2d353f52487421db684a71b1865ef81ab --label org.opencontainers.image.revision=0e5846e2d353f52487421db684a71b1865ef81ab --label org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --label quay.expires-after=6h --label build-date=2026-04-21T18:47:57Z --label org.opencontainers.image.created=2026-04-21T18:47:57Z --annotation org.opencontainers.image.revision=0e5846e2d353f52487421db684a71b1865ef81ab --annotation org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --annotation org.opencontainers.image.created=2026-04-21T18:47:57Z --tls-verify=true --no-cache --ulimit nofile=4096:4096 --http-proxy=false -f /tmp/Dockerfile.g253Z4 -t quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-0e5846e2d353f52487421db684a71b1865ef81ab . STEP 1/10: FROM registry.access.redhat.com/ubi9/python-39:1-117.1684741281 STEP 2/10: EXPOSE 8081/tcp STEP 3/10: ENV FLASK_PORT=8081 STEP 4/10: WORKDIR /projects STEP 5/10: COPY . . STEP 6/10: RUN if [ -f requirements.txt ]; then pip install -r requirements.txt; elif [ `ls -1q *.txt | wc -l` == 1 ]; then pip install -r *.txt; fi Collecting Flask==2.3.3 Downloading flask-2.3.3-py3-none-any.whl (96 kB) Collecting Jinja2>=3.1.2 Downloading jinja2-3.1.6-py3-none-any.whl (134 kB) Collecting click>=8.1.3 Downloading click-8.1.8-py3-none-any.whl (98 kB) Collecting Werkzeug>=2.3.7 Downloading werkzeug-3.1.8-py3-none-any.whl (226 kB) Collecting itsdangerous>=2.1.2 Downloading itsdangerous-2.2.0-py3-none-any.whl (16 kB) Collecting blinker>=1.6.2 Downloading blinker-1.9.0-py3-none-any.whl (8.5 kB) Collecting importlib-metadata>=3.6.0 Downloading importlib_metadata-8.7.1-py3-none-any.whl (27 kB) Collecting zipp>=3.20 Downloading zipp-3.23.1-py3-none-any.whl (10 kB) Collecting MarkupSafe>=2.0 Downloading markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB) Installing collected packages: zipp, MarkupSafe, Werkzeug, Jinja2, itsdangerous, importlib-metadata, click, blinker, Flask Successfully installed Flask-2.3.3 Jinja2-3.1.6 MarkupSafe-3.0.3 Werkzeug-3.1.8 blinker-1.9.0 click-8.1.8 importlib-metadata-8.7.1 itsdangerous-2.2.0 zipp-3.23.1 WARNING: You are using pip version 21.3.1; however, version 26.0.1 is available. You should consider upgrading via the '/opt/app-root/bin/python3.9 -m pip install --upgrade pip' command. STEP 7/10: CMD [ "python", "./app.py" ] STEP 8/10: COPY labels.json /usr/share/buildinfo/labels.json STEP 9/10: COPY labels.json /root/buildinfo/labels.json STEP 10/10: LABEL "architecture"="x86_64" "vcs-type"="git" "vcs-ref"="0e5846e2d353f52487421db684a71b1865ef81ab" "org.opencontainers.image.revision"="0e5846e2d353f52487421db684a71b1865ef81ab" "org.opencontainers.image.source"="https://github.com/redhat-appstudio-qe/sample-multi-component" "quay.expires-after"="6h" "build-date"="2026-04-21T18:47:57Z" "org.opencontainers.image.created"="2026-04-21T18:47:57Z" COMMIT quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-0e5846e2d353f52487421db684a71b1865ef81ab --> 076ed8b2bb5c Successfully tagged quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-0e5846e2d353f52487421db684a71b1865ef81ab 076ed8b2bb5c65586b9dc9fd737acea90af7709ef630042d30d149c81e0d6130 [2026-04-21T18:49:32,191827494+00:00] Unsetting proxy [2026-04-21T18:49:32,193122002+00:00] Add metadata Recording base image digests used registry.access.redhat.com/ubi9/python-39:1-117.1684741281 registry.access.redhat.com/ubi9/python-39:1-117.1684741281@sha256:40a58935b9c22664927b22bf256f53a3d744ddb7316f3af18061099e199526ee Getting image source signatures Copying blob sha256:2ecea419c65da8d685ac8b47eaec5e83290cef2fabab08c48f8cc41217cb15d1 Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde Copying config sha256:076ed8b2bb5c65586b9dc9fd737acea90af7709ef630042d30d149c81e0d6130 Writing manifest to image destination [2026-04-21T18:50:19,832105830+00:00] End build pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | container step-push: [2026-04-21T18:50:20,114931948+00:00] Update CA trust INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:50:42,920903609+00:00] Convert image [2026-04-21T18:50:42,922185915+00:00] Push image with unique tag Pushing to quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-62np9-build-container [retry] executing: buildah push --format=docker --retry 3 --tls-verify=true quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-0e5846e2d353f52487421db684a71b1865ef81ab docker://quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-62np9-build-container Getting image source signatures Copying blob sha256:2ecea419c65da8d685ac8b47eaec5e83290cef2fabab08c48f8cc41217cb15d1 Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | container step-sbom-syft-generate: pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | container step-prepare-sboms: pod: python-component-sjporj-on-a36e92adecccf5e83d0e472057f299e3-pod | container step-upload-sbom: pod: python-component-sjporj-on-ce13004aef26f7f945dd899b768f1e15-pod | init container: prepare 2026/04/21 18:46:57 Entrypoint initialization pod: python-component-sjporj-on-ce13004aef26f7f945dd899b768f1e15-pod | init container: place-scripts 2026/04/21 18:46:57 Decoded script /tekton/scripts/script-0-54gxr 2026/04/21 18:46:57 Decoded script /tekton/scripts/script-1-hscvr pod: python-component-sjporj-on-ce13004aef26f7f945dd899b768f1e15-pod | container step-clone: INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt {"level":"info","ts":1776797221.4412205,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776797221.7028315,"caller":"git/git.go:223","msg":"Successfully cloned https://github.com/redhat-appstudio-qe/sample-multi-component @ 0e5846e2d353f52487421db684a71b1865ef81ab (grafted, HEAD) in path /var/workdir/source"} {"level":"info","ts":1776797221.70288,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776797221.7291105,"caller":"git/git.go:277","msg":"Successfully initialized and updated submodules in path /var/workdir/source"} Merge option disabled. Using checked-out revision 0e5846e2d353f52487421db684a71b1865ef81ab directly. pod: python-component-sjporj-on-ce13004aef26f7f945dd899b768f1e15-pod | container step-symlink-check: Running symlink check pod: python-component-sjporj-on-ce13004aef26f7f945dd899b768f1e15-pod | container step-create-trusted-artifact: Prepared artifact from /var/workdir/source (sha256:47e1aac606956ea676ecf6d0943ca92a00c8566946c44fb0b30f5cdbb27bea40) Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras push --annotation=quay.expires-after=6h --registry-config /tmp/create-oci.sh.7FHizw/auth-13Fqrv.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-0e5846e2d353f52487421db684a71b1865ef81ab.git SOURCE_ARTIFACT Uploading 47e1aac60695 SOURCE_ARTIFACT Uploaded 47e1aac60695 SOURCE_ARTIFACT Pushed [registry] quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-0e5846e2d353f52487421db684a71b1865ef81ab.git ArtifactType: application/vnd.unknown.artifact.v1 Digest: sha256:0105bce944b1e0cbda50eb2435bad97dc16f9605fdeb2e0bf4cd964a2caee732 Artifacts created pod: python-component-sjporj-on-pull-request-62np9-init-pod | init container: prepare 2026/04/21 18:46:52 Entrypoint initialization pod: python-component-sjporj-on-pull-request-62np9-init-pod | container step-init: time="2026-04-21T18:46:54Z" level=info msg="Using in-cluster config" logger=KubeClient time="2026-04-21T18:46:54Z" level=info msg="[param] enable: false" time="2026-04-21T18:46:54Z" level=info msg="[param] default-http-proxy: squid.caching.svc.cluster.local:3128" time="2026-04-21T18:46:54Z" level=info msg="[param] default-no-proxy: brew.registry.redhat.io,docker.io,gcr.io,ghcr.io,images.paas.redhat.com,mirror.gcr.io,nvcr.io,quay.io,registry-proxy.engineering.redhat.com,registry.access.redhat.com,registry.ci.openshift.org,registry.fedoraproject.org,registry.redhat.io,registry.stage.redhat.io,vault.habana.ai" time="2026-04-21T18:46:54Z" level=info msg="[param] http-proxy-result-path: /tekton/results/http-proxy" time="2026-04-21T18:46:54Z" level=info msg="[param] no-proxy-result-path: /tekton/results/no-proxy" time="2026-04-21T18:46:54Z" level=info msg="Cache proxy is disabled in param or in backend" time="2026-04-21T18:46:54Z" level=info msg="[result] HTTP PROXY: " time="2026-04-21T18:46:54Z" level=info msg="[result] NO PROXY: " New PipelineRun python-component-sjporj-on-pull-request-hk4l7 found after retrigger for component build-e2e-uurm/python-component-sjporj PipelineRun python-component-sjporj-on-pull-request-hk4l7 found for Component build-e2e-uurm/python-component-sjporj PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Running PipelineRun python-component-sjporj-on-pull-request-hk4l7 reason: Failed attempt 3/3: PipelineRun "python-component-sjporj-on-pull-request-hk4l7" failed: pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | init container: prepare 2026/04/21 18:53:28 Entrypoint initialization pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | init container: place-scripts 2026/04/21 18:53:28 Decoded script /tekton/scripts/script-0-pj55w 2026/04/21 18:53:28 Decoded script /tekton/scripts/script-1-zjljz pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-clone: INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt {"level":"info","ts":1776797612.9591951,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776797613.5570908,"caller":"git/git.go:223","msg":"Successfully cloned https://github.com/redhat-appstudio-qe/sample-multi-component @ 08ea948bbad920b0cb7ec57845ff652ab9610998 (grafted, HEAD) in path /var/workdir/source"} {"level":"info","ts":1776797613.5571344,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776797613.5815067,"caller":"git/git.go:277","msg":"Successfully initialized and updated submodules in path /var/workdir/source"} Merge option disabled. Using checked-out revision 08ea948bbad920b0cb7ec57845ff652ab9610998 directly. pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-symlink-check: Running symlink check pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-create-trusted-artifact: Prepared artifact from /var/workdir/source (sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f) Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras push --annotation=quay.expires-after=6h --registry-config /tmp/create-oci.sh.pq8C6z/auth-JzbxqA.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998.git SOURCE_ARTIFACT Uploading 2a49833ad238 SOURCE_ARTIFACT Uploaded 2a49833ad238 SOURCE_ARTIFACT Pushed [registry] quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998.git ArtifactType: application/vnd.unknown.artifact.v1 Digest: sha256:456412b2e8ae798284452dd94c4500100d291df3122819fcc4e734ff23aecfb8 Artifacts created pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | init container: prepare 2026/04/21 18:54:23 Entrypoint initialization pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | init container: place-scripts 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-1-45lnk 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-2-ljtm4 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-3-h25tm 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-4-nqcqb 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-5-79cdj pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-use-trusted-artifact: Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras blob fetch --registry-config /tmp/use-oci.sh.PEagk7/auth-FHOyky.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f --output - Restored artifact quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f to /var/workdir/source WARN: artifact URI not provided, (given: =/var/workdir/cachi2) pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-build: [2026-04-21T18:54:29,815772926+00:00] Validate context path [2026-04-21T18:54:29,819168435+00:00] Update CA trust [2026-04-21T18:54:29,820338880+00:00] Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:54:34,059059880+00:00] Prepare Dockerfile Checking if /var/workdir/cachi2/output/bom.json exists. Could not find prefetched sbom. No content_sets found for ICM [2026-04-21T18:54:34,065269108+00:00] Prepare system (architecture: x86_64) [2026-04-21T18:54:34,126797900+00:00] Setup prefetched Trying to pull registry.access.redhat.com/ubi9/python-39:1-117.1684741281... Getting image source signatures Checking if image destination supports signatures Copying blob sha256:8a2e9815396eaa57b0dfe27b2dfc442417f06aa70974c358f198980b533c7942 Copying blob sha256:5465449a2ea5d8a7e05fa498c8ba35f0ce7714b0ad37e716961fca17dfa1ef13 Copying blob sha256:085efe85e9dfb0b3d5d2392e4b78660a3091bf1ded336315117a627fbdd0567d Copying blob sha256:a07d808ddf4404b3885fef6100142a973cc667b2c3c4abcf4db96508795b1efb Copying config sha256:45c8f23c11bb64f238236aaf5cbe05595778e1d4fb6c06109c6fcc0a6b57a45b Writing manifest to image destination Storing signatures [2026-04-21T18:55:12,571935025+00:00] Unsetting proxy { "architecture": "x86_64", "build-date": "2026-04-21T18:54:34Z", "com.redhat.component": "python-39-container", "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI", "description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "distribution-scope": "public", "io.buildah.version": "1.42.2", "io.buildpacks.stack.id": "com.redhat.stacks.ubi9-python-39", "io.k8s.description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "io.k8s.display-name": "Python 3.9", "io.openshift.expose-services": "8080:http", "io.openshift.s2i.scripts-url": "image:///usr/libexec/s2i", "io.openshift.tags": "builder,python,python39,python-39,rh-python39", "io.s2i.scripts-url": "image:///usr/libexec/s2i", "maintainer": "SoftwareCollections.org ", "name": "ubi9/python-39", "release": "117.1684741281", "summary": "Platform for building and running Python 3.9 applications", "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/python-39/images/1-117.1684741281", "usage": "s2i build https://github.com/sclorg/s2i-python-container.git --context-dir=3.9/test/setup-test-app/ ubi9/python-39 python-sample-app", "vcs-ref": "08ea948bbad920b0cb7ec57845ff652ab9610998", "vcs-type": "git", "vendor": "Red Hat, Inc.", "version": "1", "org.opencontainers.image.revision": "08ea948bbad920b0cb7ec57845ff652ab9610998", "org.opencontainers.image.source": "https://github.com/redhat-appstudio-qe/sample-multi-component", "quay.expires-after": "6h", "org.opencontainers.image.created": "2026-04-21T18:54:34Z" } [2026-04-21T18:55:12,695221494+00:00] Register sub-man Adding the entitlement to the build [2026-04-21T18:55:12,698688196+00:00] Add secrets [2026-04-21T18:55:12,706294828+00:00] Run buildah build [2026-04-21T18:55:12,721298123+00:00] buildah build --volume /tmp/entitlement:/etc/pki/entitlement --security-opt=unmask=/proc/interrupts --label architecture=x86_64 --label vcs-type=git --label vcs-ref=08ea948bbad920b0cb7ec57845ff652ab9610998 --label org.opencontainers.image.revision=08ea948bbad920b0cb7ec57845ff652ab9610998 --label org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --label quay.expires-after=6h --label build-date=2026-04-21T18:54:34Z --label org.opencontainers.image.created=2026-04-21T18:54:34Z --annotation org.opencontainers.image.revision=08ea948bbad920b0cb7ec57845ff652ab9610998 --annotation org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --annotation org.opencontainers.image.created=2026-04-21T18:54:34Z --tls-verify=true --no-cache --ulimit nofile=4096:4096 --http-proxy=false -f /tmp/Dockerfile.UhJu8u -t quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 . STEP 1/10: FROM registry.access.redhat.com/ubi9/python-39:1-117.1684741281 STEP 2/10: EXPOSE 8081/tcp STEP 3/10: ENV FLASK_PORT=8081 STEP 4/10: WORKDIR /projects STEP 5/10: COPY . . STEP 6/10: RUN if [ -f requirements.txt ]; then pip install -r requirements.txt; elif [ `ls -1q *.txt | wc -l` == 1 ]; then pip install -r *.txt; fi Collecting Flask==2.3.3 Downloading flask-2.3.3-py3-none-any.whl (96 kB) Collecting blinker>=1.6.2 Downloading blinker-1.9.0-py3-none-any.whl (8.5 kB) Collecting click>=8.1.3 Downloading click-8.1.8-py3-none-any.whl (98 kB) Collecting Jinja2>=3.1.2 Downloading jinja2-3.1.6-py3-none-any.whl (134 kB) Collecting Werkzeug>=2.3.7 Downloading werkzeug-3.1.8-py3-none-any.whl (226 kB) Collecting itsdangerous>=2.1.2 Downloading itsdangerous-2.2.0-py3-none-any.whl (16 kB) Collecting importlib-metadata>=3.6.0 Downloading importlib_metadata-8.7.1-py3-none-any.whl (27 kB) Collecting zipp>=3.20 Downloading zipp-3.23.1-py3-none-any.whl (10 kB) Collecting MarkupSafe>=2.0 Downloading markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB) Installing collected packages: zipp, MarkupSafe, Werkzeug, Jinja2, itsdangerous, importlib-metadata, click, blinker, Flask Successfully installed Flask-2.3.3 Jinja2-3.1.6 MarkupSafe-3.0.3 Werkzeug-3.1.8 blinker-1.9.0 click-8.1.8 importlib-metadata-8.7.1 itsdangerous-2.2.0 zipp-3.23.1 WARNING: You are using pip version 21.3.1; however, version 26.0.1 is available. You should consider upgrading via the '/opt/app-root/bin/python3.9 -m pip install --upgrade pip' command. STEP 7/10: CMD [ "python", "./app.py" ] STEP 8/10: COPY labels.json /usr/share/buildinfo/labels.json STEP 9/10: COPY labels.json /root/buildinfo/labels.json STEP 10/10: LABEL "architecture"="x86_64" "vcs-type"="git" "vcs-ref"="08ea948bbad920b0cb7ec57845ff652ab9610998" "org.opencontainers.image.revision"="08ea948bbad920b0cb7ec57845ff652ab9610998" "org.opencontainers.image.source"="https://github.com/redhat-appstudio-qe/sample-multi-component" "quay.expires-after"="6h" "build-date"="2026-04-21T18:54:34Z" "org.opencontainers.image.created"="2026-04-21T18:54:34Z" COMMIT quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 --> 34b244364882 Successfully tagged quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 34b2443648828a1aef85cf3f4e2ae808c8febd22d41f5728c3bf492894162a59 [2026-04-21T18:55:19,624080289+00:00] Unsetting proxy [2026-04-21T18:55:19,625400179+00:00] Add metadata Recording base image digests used registry.access.redhat.com/ubi9/python-39:1-117.1684741281 registry.access.redhat.com/ubi9/python-39:1-117.1684741281@sha256:40a58935b9c22664927b22bf256f53a3d744ddb7316f3af18061099e199526ee Getting image source signatures Copying blob sha256:eef098756a3e3d8f1cd4e99ce529aac3f2fb922bc8143fdf97dbbfeec1cf7565 Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde Copying config sha256:34b2443648828a1aef85cf3f4e2ae808c8febd22d41f5728c3bf492894162a59 Writing manifest to image destination [2026-04-21T18:56:05,125408972+00:00] End build pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-push: [2026-04-21T18:56:05,973214425+00:00] Update CA trust INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:56:28,425715986+00:00] Convert image [2026-04-21T18:56:28,426803303+00:00] Push image with unique tag Pushing to quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-hk4l7-build-container [retry] executing: buildah push --format=docker --retry 3 --tls-verify=true quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 docker://quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-hk4l7-build-container Getting image source signatures Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:eef098756a3e3d8f1cd4e99ce529aac3f2fb922bc8143fdf97dbbfeec1cf7565 pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-sbom-syft-generate: pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-prepare-sboms: pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-upload-sbom: pod: python-component-sjporj-on-pull-request-hk4l7-init-pod | init container: prepare 2026/04/21 18:53:23 Entrypoint initialization pod: python-component-sjporj-on-pull-request-hk4l7-init-pod | container step-init: time="2026-04-21T18:53:26Z" level=info msg="Using in-cluster config" logger=KubeClient time="2026-04-21T18:53:26Z" level=info msg="[param] enable: false" time="2026-04-21T18:53:26Z" level=info msg="[param] default-http-proxy: squid.caching.svc.cluster.local:3128" time="2026-04-21T18:53:26Z" level=info msg="[param] default-no-proxy: brew.registry.redhat.io,docker.io,gcr.io,ghcr.io,images.paas.redhat.com,mirror.gcr.io,nvcr.io,quay.io,registry-proxy.engineering.redhat.com,registry.access.redhat.com,registry.ci.openshift.org,registry.fedoraproject.org,registry.redhat.io,registry.stage.redhat.io,vault.habana.ai" time="2026-04-21T18:53:26Z" level=info msg="[param] http-proxy-result-path: /tekton/results/http-proxy" time="2026-04-21T18:53:26Z" level=info msg="[param] no-proxy-result-path: /tekton/results/no-proxy" time="2026-04-21T18:53:26Z" level=info msg="Cache proxy is disabled in param or in backend" time="2026-04-21T18:53:26Z" level=info msg="[result] HTTP PROXY: " time="2026-04-21T18:53:26Z" level=info msg="[result] NO PROXY: " [FAILED] in [It] - /tmp/tmp.dle8pz85Yc/tests/build/multi_component.go:161 @ 04/21/26 18:58:43.107 << Timeline [FAILED] Expected success, but got an error: <*errors.errorString | 0xc000868de0>: pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | init container: prepare 2026/04/21 18:53:28 Entrypoint initialization pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | init container: place-scripts 2026/04/21 18:53:28 Decoded script /tekton/scripts/script-0-pj55w 2026/04/21 18:53:28 Decoded script /tekton/scripts/script-1-zjljz pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-clone: INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt {"level":"info","ts":1776797612.9591951,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776797613.5570908,"caller":"git/git.go:223","msg":"Successfully cloned https://github.com/redhat-appstudio-qe/sample-multi-component @ 08ea948bbad920b0cb7ec57845ff652ab9610998 (grafted, HEAD) in path /var/workdir/source"} {"level":"info","ts":1776797613.5571344,"caller":"git/git.go:394","msg":"Retrying operation (attempt 1)"} {"level":"info","ts":1776797613.5815067,"caller":"git/git.go:277","msg":"Successfully initialized and updated submodules in path /var/workdir/source"} Merge option disabled. Using checked-out revision 08ea948bbad920b0cb7ec57845ff652ab9610998 directly. pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-symlink-check: Running symlink check pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-create-trusted-artifact: Prepared artifact from /var/workdir/source (sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f) Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras push --annotation=quay.expires-after=6h --registry-config /tmp/create-oci.sh.pq8C6z/auth-JzbxqA.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998.git SOURCE_ARTIFACT Uploading 2a49833ad238 SOURCE_ARTIFACT Uploaded 2a49833ad238 SOURCE_ARTIFACT Pushed [registry] quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998.git ArtifactType: application/vnd.unknown.artifact.v1 Digest: sha256:456412b2e8ae798284452dd94c4500100d291df3122819fcc4e734ff23aecfb8 Artifacts created pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | init container: prepare 2026/04/21 18:54:23 Entrypoint initialization pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | init container: place-scripts 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-1-45lnk 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-2-ljtm4 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-3-h25tm 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-4-nqcqb 2026/04/21 18:54:24 Decoded script /tekton/scripts/script-5-79cdj pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-use-trusted-artifact: Using token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj Executing: oras blob fetch --registry-config /tmp/use-oci.sh.PEagk7/auth-FHOyky.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f --output - Restored artifact quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f to /var/workdir/source WARN: artifact URI not provided, (given: =/var/workdir/cachi2) pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-build: [2026-04-21T18:54:29,815772926+00:00] Validate context path [2026-04-21T18:54:29,819168435+00:00] Update CA trust [2026-04-21T18:54:29,820338880+00:00] Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:54:34,059059880+00:00] Prepare Dockerfile Checking if /var/workdir/cachi2/output/bom.json exists. Could not find prefetched sbom. No content_sets found for ICM [2026-04-21T18:54:34,065269108+00:00] Prepare system (architecture: x86_64) [2026-04-21T18:54:34,126797900+00:00] Setup prefetched Trying to pull registry.access.redhat.com/ubi9/python-39:1-117.1684741281... Getting image source signatures Checking if image destination supports signatures Copying blob sha256:8a2e9815396eaa57b0dfe27b2dfc442417f06aa70974c358f198980b533c7942 Copying blob sha256:5465449a2ea5d8a7e05fa498c8ba35f0ce7714b0ad37e716961fca17dfa1ef13 Copying blob sha256:085efe85e9dfb0b3d5d2392e4b78660a3091bf1ded336315117a627fbdd0567d Copying blob sha256:a07d808ddf4404b3885fef6100142a973cc667b2c3c4abcf4db96508795b1efb Copying config sha256:45c8f23c11bb64f238236aaf5cbe05595778e1d4fb6c06109c6fcc0a6b57a45b Writing manifest to image destination Storing signatures [2026-04-21T18:55:12,571935025+00:00] Unsetting proxy { "architecture": "x86_64", "build-date": "2026-04-21T18:54:34Z", "com.redhat.component": "python-39-container", "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI", "description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "distribution-scope": "public", "io.buildah.version": "1.42.2", "io.buildpacks.stack.id": "com.redhat.stacks.ubi9-python-39", "io.k8s.description": "Python 3.9 available as container is a base platform for building and running various Python 3.9 applications and frameworks. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.", "io.k8s.display-name": "Python 3.9", "io.openshift.expose-services": "8080:http", "io.openshift.s2i.scripts-url": "image:///usr/libexec/s2i", "io.openshift.tags": "builder,python,python39,python-39,rh-python39", "io.s2i.scripts-url": "image:///usr/libexec/s2i", "maintainer": "SoftwareCollections.org ", "name": "ubi9/python-39", "release": "117.1684741281", "summary": "Platform for building and running Python 3.9 applications", "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/python-39/images/1-117.1684741281", "usage": "s2i build https://github.com/sclorg/s2i-python-container.git --context-dir=3.9/test/setup-test-app/ ubi9/python-39 python-sample-app", "vcs-ref": "08ea948bbad920b0cb7ec57845ff652ab9610998", "vcs-type": "git", "vendor": "Red Hat, Inc.", "version": "1", "org.opencontainers.image.revision": "08ea948bbad920b0cb7ec57845ff652ab9610998", "org.opencontainers.image.source": "https://github.com/redhat-appstudio-qe/sample-multi-component", "quay.expires-after": "6h", "org.opencontainers.image.created": "2026-04-21T18:54:34Z" } [2026-04-21T18:55:12,695221494+00:00] Register sub-man Adding the entitlement to the build [2026-04-21T18:55:12,698688196+00:00] Add secrets [2026-04-21T18:55:12,706294828+00:00] Run buildah build [2026-04-21T18:55:12,721298123+00:00] buildah build --volume /tmp/entitlement:/etc/pki/entitlement --security-opt=unmask=/proc/interrupts --label architecture=x86_64 --label vcs-type=git --label vcs-ref=08ea948bbad920b0cb7ec57845ff652ab9610998 --label org.opencontainers.image.revision=08ea948bbad920b0cb7ec57845ff652ab9610998 --label org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --label quay.expires-after=6h --label build-date=2026-04-21T18:54:34Z --label org.opencontainers.image.created=2026-04-21T18:54:34Z --annotation org.opencontainers.image.revision=08ea948bbad920b0cb7ec57845ff652ab9610998 --annotation org.opencontainers.image.source=https://github.com/redhat-appstudio-qe/sample-multi-component --annotation org.opencontainers.image.created=2026-04-21T18:54:34Z --tls-verify=true --no-cache --ulimit nofile=4096:4096 --http-proxy=false -f /tmp/Dockerfile.UhJu8u -t quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 . STEP 1/10: FROM registry.access.redhat.com/ubi9/python-39:1-117.1684741281 STEP 2/10: EXPOSE 8081/tcp STEP 3/10: ENV FLASK_PORT=8081 STEP 4/10: WORKDIR /projects STEP 5/10: COPY . . STEP 6/10: RUN if [ -f requirements.txt ]; then pip install -r requirements.txt; elif [ `ls -1q *.txt | wc -l` == 1 ]; then pip install -r *.txt; fi Collecting Flask==2.3.3 Downloading flask-2.3.3-py3-none-any.whl (96 kB) Collecting blinker>=1.6.2 Downloading blinker-1.9.0-py3-none-any.whl (8.5 kB) Collecting click>=8.1.3 Downloading click-8.1.8-py3-none-any.whl (98 kB) Collecting Jinja2>=3.1.2 Downloading jinja2-3.1.6-py3-none-any.whl (134 kB) Collecting Werkzeug>=2.3.7 Downloading werkzeug-3.1.8-py3-none-any.whl (226 kB) Collecting itsdangerous>=2.1.2 Downloading itsdangerous-2.2.0-py3-none-any.whl (16 kB) Collecting importlib-metadata>=3.6.0 Downloading importlib_metadata-8.7.1-py3-none-any.whl (27 kB) Collecting zipp>=3.20 Downloading zipp-3.23.1-py3-none-any.whl (10 kB) Collecting MarkupSafe>=2.0 Downloading markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB) Installing collected packages: zipp, MarkupSafe, Werkzeug, Jinja2, itsdangerous, importlib-metadata, click, blinker, Flask Successfully installed Flask-2.3.3 Jinja2-3.1.6 MarkupSafe-3.0.3 Werkzeug-3.1.8 blinker-1.9.0 click-8.1.8 importlib-metadata-8.7.1 itsdangerous-2.2.0 zipp-3.23.1 WARNING: You are using pip version 21.3.1; however, version 26.0.1 is available. You should consider upgrading via the '/opt/app-root/bin/python3.9 -m pip install --upgrade pip' command. STEP 7/10: CMD [ "python", "./app.py" ] STEP 8/10: COPY labels.json /usr/share/buildinfo/labels.json STEP 9/10: COPY labels.json /root/buildinfo/labels.json STEP 10/10: LABEL "architecture"="x86_64" "vcs-type"="git" "vcs-ref"="08ea948bbad920b0cb7ec57845ff652ab9610998" "org.opencontainers.image.revision"="08ea948bbad920b0cb7ec57845ff652ab9610998" "org.opencontainers.image.source"="https://github.com/redhat-appstudio-qe/sample-multi-component" "quay.expires-after"="6h" "build-date"="2026-04-21T18:54:34Z" "org.opencontainers.image.created"="2026-04-21T18:54:34Z" COMMIT quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 --> 34b244364882 Successfully tagged quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 34b2443648828a1aef85cf3f4e2ae808c8febd22d41f5728c3bf492894162a59 [2026-04-21T18:55:19,624080289+00:00] Unsetting proxy [2026-04-21T18:55:19,625400179+00:00] Add metadata Recording base image digests used registry.access.redhat.com/ubi9/python-39:1-117.1684741281 registry.access.redhat.com/ubi9/python-39:1-117.1684741281@sha256:40a58935b9c22664927b22bf256f53a3d744ddb7316f3af18061099e199526ee Getting image source signatures Copying blob sha256:eef098756a3e3d8f1cd4e99ce529aac3f2fb922bc8143fdf97dbbfeec1cf7565 Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde Copying config sha256:34b2443648828a1aef85cf3f4e2ae808c8febd22d41f5728c3bf492894162a59 Writing manifest to image destination [2026-04-21T18:56:05,125408972+00:00] End build pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-push: [2026-04-21T18:56:05,973214425+00:00] Update CA trust INFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt '/mnt/trusted-ca/ca-bundle.crt' -> '/etc/pki/ca-trust/source/anchors/ca-bundle.crt' [2026-04-21T18:56:28,425715986+00:00] Convert image [2026-04-21T18:56:28,426803303+00:00] Push image with unique tag Pushing to quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-hk4l7-build-container [retry] executing: buildah push --format=docker --retry 3 --tls-verify=true quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998 docker://quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:python-component-sjporj-on-pull-request-hk4l7-build-container Getting image source signatures Copying blob sha256:d661a108c347030c3ef626a0e0020e084fba1e1023193824f5395fa90ef40bde Copying blob sha256:629980bfa23ec05ce5acc61ed2866ddf8713f40bb2a61f0850395082f1457220 Copying blob sha256:bc8a0003b3470be53e5cf295daf38c287d3f20ab90e8f5c2b6f17406b158d323 Copying blob sha256:d93d3cc0bc0c5ed9655bcdcf31183ed0b570076c0e6a13e0d7cb907d0b877b00 Copying blob sha256:eef098756a3e3d8f1cd4e99ce529aac3f2fb922bc8143fdf97dbbfeec1cf7565 pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-sbom-syft-generate: pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-prepare-sboms: pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-upload-sbom: pod: python-component-sjporj-on-pull-request-hk4l7-init-pod | init container: prepare 2026/04/21 18:53:23 Entrypoint initialization pod: python-component-sjporj-on-pull-request-hk4l7-init-pod | container step-init: time="2026-04-21T18:53:26Z" level=info msg="Using in-cluster config" logger=KubeClient time="2026-04-21T18:53:26Z" level=info msg="[param] enable: false" time="2026-04-21T18:53:26Z" level=info msg="[param] default-http-proxy: squid.caching.svc.cluster.local:3128" time="2026-04-21T18:53:26Z" level=info msg="[param] default-no-proxy: brew.registry.redhat.io,docker.io,gcr.io,ghcr.io,images.paas.redhat.com,mirror.gcr.io,nvcr.io,quay.io,registry-proxy.engineering.redhat.com,registry.access.redhat.com,registry.ci.openshift.org,registry.fedoraproject.org,registry.redhat.io,registry.stage.redhat.io,vault.habana.ai" time="2026-04-21T18:53:26Z" level=info msg="[param] http-proxy-result-path: /tekton/results/http-proxy" time="2026-04-21T18:53:26Z" level=info msg="[param] no-proxy-result-path: /tekton/results/no-proxy" time="2026-04-21T18:53:26Z" level=info msg="Cache proxy is disabled in param or in backend" time="2026-04-21T18:53:26Z" level=info msg="[result] HTTP PROXY: " time="2026-04-21T18:53:26Z" level=info msg="[result] NO PROXY: " { s: "\n pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | init container: prepare\n2026/04/21 18:53:28 Entrypoint initialization\n\n pod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | init container: place-scripts\n2026/04/21 18:53:28 Decoded script /tekton/scripts/script-0-pj55w\n2026/04/21 18:53:28 Decoded script /tekton/scripts/script-1-zjljz\n\npod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-clone: \nINFO: Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt\n{\"level\":\"info\",\"ts\":1776797612.9591951,\"caller\":\"git/git.go:394\",\"msg\":\"Retrying operation (attempt 1)\"}\n{\"level\":\"info\",\"ts\":1776797613.5570908,\"caller\":\"git/git.go:223\",\"msg\":\"Successfully cloned https://github.com/redhat-appstudio-qe/sample-multi-component @ 08ea948bbad920b0cb7ec57845ff652ab9610998 (grafted, HEAD) in path /var/workdir/source\"}\n{\"level\":\"info\",\"ts\":1776797613.5571344,\"caller\":\"git/git.go:394\",\"msg\":\"Retrying operation (attempt 1)\"}\n{\"level\":\"info\",\"ts\":1776797613.5815067,\"caller\":\"git/git.go:277\",\"msg\":\"Successfully initialized and updated submodules in path /var/workdir/source\"}\nMerge option disabled. Using checked-out revision 08ea948bbad920b0cb7ec57845ff652ab9610998 directly.\n\npod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-symlink-check: \nRunning symlink check\n\npod: python-component-sjporj-on-56d03711a058adfebc86cd5fe98484f3-pod | container step-create-trusted-artifact: \nPrepared artifact from /var/workdir/source (sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f)\nUsing token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj\nExecuting: oras push --annotation=quay.expires-after=6h --registry-config /tmp/create-oci.sh.pq8C6z/auth-JzbxqA.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998.git SOURCE_ARTIFACT\nUploading 2a49833ad238 SOURCE_ARTIFACT\nUploaded 2a49833ad238 SOURCE_ARTIFACT\nPushed [registry] quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj:on-pr-08ea948bbad920b0cb7ec57845ff652ab9610998.git\nArtifactType: application/vnd.unknown.artifact.v1\nDigest: sha256:456412b2e8ae798284452dd94c4500100d291df3122819fcc4e734ff23aecfb8\nArtifacts created\n\n\n pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | init container: prepare\n2026/04/21 18:54:23 Entrypoint initialization\n\n pod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | init container: place-scripts\n2026/04/21 18:54:24 Decoded script /tekton/scripts/script-1-45lnk\n2026/04/21 18:54:24 Decoded script /tekton/scripts/script-2-ljtm4\n2026/04/21 18:54:24 Decoded script /tekton/scripts/script-3-h25tm\n2026/04/21 18:54:24 Decoded script /tekton/scripts/script-4-nqcqb\n2026/04/21 18:54:24 Decoded script /tekton/scripts/script-5-79cdj\n\npod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-use-trusted-artifact: \nUsing token for quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj\nExecuting: oras blob fetch --registry-config /tmp/use-oci.sh.PEagk7/auth-FHOyky.json quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f --output -\nRestored artifact quay.io/redhat-appstudio-qe/build-e2e-uurm/python-component-sjporj@sha256:2a49833ad2384d11018af905320562b6e4f03b954db6af6cfb642d116a6eb69f to /var/workdir/source\nWARN: artifact URI not provided, (given: =/var/workdir/cachi2)\n\n\npod: python-component-sjporj-on-f0a2be294464a2c4f0070cf7e0b095ee-pod | container step-build: \n[2026-04-21T18:54:29,815772926+00:00] Validate context path\n[2026-04-21T18:54:29,819168435+00:00] Update CA trust\n[2026-04-21T18:54:29,820338880+00:00] Using mounted CA bundle: /mnt/trusted-ca/ca-bundle.crt\n'/mnt/trusted-ca/ca-bundle.crt' ->... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output In [It] at: /tmp/tmp.dle8pz85Yc/tests/build/multi_component.go:161 @ 04/21/26 18:58:43.107 ------------------------------ SSSSS• ------------------------------ • [TIMEDOUT] [1078.939 seconds] [disaster-recovery DR Same-Version Backup/Restore] when creating tenants and running initial pipelines [It] should wait for all build PipelineRuns to succeed [disaster-recovery, Serial] /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/dr_same_version.go:60 Timeline >> STEP: Waiting for per-component build → test chains across all tenants @ 04/21/26 18:58:55.039 STEP: Waiting for build PipelineRun for mathwizz-frontend in dr-test-moshekipod-same-version-dr (base: 0) @ 04/21/26 18:58:55.039 STEP: Waiting for build PipelineRun for mathwizz-web-server in dr-test-moshekipod-same-version-dr (base: 0) @ 04/21/26 18:58:55.039 STEP: Waiting for build PipelineRun for mathwizz-frontend in dr-test-kokohazamar-same-version-dr (base: 0) @ 04/21/26 18:58:55.039 STEP: Waiting for build PipelineRun for mathwizz-history-worker in dr-test-kokohazamar-same-version-dr (base: 0) @ 04/21/26 18:58:55.039 STEP: Waiting for build PipelineRun for mathwizz-history-worker in dr-test-moshekipod-same-version-dr (base: 0) @ 04/21/26 18:58:55.039 STEP: Waiting for build PipelineRun for mathwizz-web-server in dr-test-kokohazamar-same-version-dr (base: 0) @ 04/21/26 18:58:55.039 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 1/1 build PipelineRuns succeeded STEP: Waiting for test PipelineRun for mathwizz-frontend in dr-test-moshekipod-same-version-dr (base: 0) @ 04/21/26 19:03:55.396 FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 1/1 build PipelineRuns succeeded STEP: Waiting for test PipelineRun for mathwizz-frontend in dr-test-kokohazamar-same-version-dr (base: 0) @ 04/21/26 19:04:25.321 namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-v5xnw (component: mathwizz-history-worker) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-lm84h (component: mathwizz-web-server) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded namespace dr-test-kokohazamar-same-version-dr: 0/1 test PipelineRuns succeeded namespace dr-test-moshekipod-same-version-dr: 0/1 test PipelineRuns succeeded FAILED build PipelineRun mathwizz-history-worker-on-pull-request-qdc5m (component: mathwizz-history-worker) in dr-test-moshekipod-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-moshekipod-same-version-dr: 0/1 build PipelineRuns succeeded FAILED build PipelineRun mathwizz-web-server-on-pull-request-cnpt7 (component: mathwizz-web-server) in dr-test-kokohazamar-same-version-dr: Tasks Completed: 15 (Failed: 1, Cancelled 0), Skipped: 1 namespace dr-test-kokohazamar-same-version-dr: 0/1 build PipelineRuns succeeded [TIMEDOUT] in [It] - /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/dr_same_version.go:60 @ 04/21/26 19:16:52.613 Deleting fork repo DR-MathWizz-vlnqjf for tenant dr-test-kokohazamar-same-version-dr Deleting fork repo DR-MathWizz-ofakoi for tenant dr-test-moshekipod-same-version-dr STEP: Collecting Velero pod information @ 04/21/26 19:16:53.942 Velero pod: node-agent-6jmdt | Phase: Running | Ready: true Velero pod: node-agent-dgrk9 | Phase: Running | Ready: true Velero pod: node-agent-hr5vs | Phase: Running | Ready: true Velero pod: velero-5498cfc4db-5dwfp | Phase: Running | Ready: true STEP: Collecting Backup CR status for tenant "dr-test-kokohazamar-same-version-dr" @ 04/21/26 19:16:53.954 WARNING: could not get Backup CR "backup-kokohazamar-same-version-dr": backups.velero.io "backup-kokohazamar-same-version-dr" not found STEP: Collecting Restore CR status for tenant "dr-test-kokohazamar-same-version-dr" @ 04/21/26 19:16:53.96 WARNING: could not get Restore CR "restore-backup-kokohazamar-same-version-dr": restores.velero.io "restore-backup-kokohazamar-same-version-dr" not found STEP: Collecting Backup CR status for tenant "dr-test-moshekipod-same-version-dr" @ 04/21/26 19:16:53.966 WARNING: could not get Backup CR "backup-moshekipod-same-version-dr": backups.velero.io "backup-moshekipod-same-version-dr" not found STEP: Collecting Restore CR status for tenant "dr-test-moshekipod-same-version-dr" @ 04/21/26 19:16:53.972 WARNING: could not get Restore CR "restore-backup-moshekipod-same-version-dr": restores.velero.io "restore-backup-moshekipod-same-version-dr" not found << Timeline [TIMEDOUT] A suite timeout occurred In [It] at: /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/dr_same_version.go:60 @ 04/21/26 19:16:52.613 This is the Progress Report generated when the suite timeout occurred: [disaster-recovery DR Same-Version Backup/Restore] when creating tenants and running initial pipelines should wait for all build PipelineRuns to succeed (Spec Runtime: 17m57.574s) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/dr_same_version.go:60 In [It] (Node Runtime: 17m57.574s) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/dr_same_version.go:60 At [By Step] Waiting for test PipelineRun for mathwizz-frontend in dr-test-kokohazamar-same-version-dr (base: 0) (Step Runtime: 12m27.292s) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:168 Spec Goroutine goroutine 683 [sync.WaitGroup.Wait, 18 minutes] sync.runtime_SemacquireWaitGroup(0xc001869540?, 0xc0?) /usr/lib/golang/src/runtime/sema.go:114 sync.(*WaitGroup).Wait(0xc001837310) /usr/lib/golang/src/sync/waitgroup.go:206 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains(0xc001a5cc00, {0xc000072cc0, 0x2, 0xc00168dea0?}, 0x0, 0x0) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:175 | } | } > wg.Wait() | | // Release PipelineRuns run in the managed namespace and may not map 1:1 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.init.func2.2.2() /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/dr_same_version.go:61 | | It("should wait for all build PipelineRuns to succeed", func() { > waitForPipelineChains(fw, svTenants, nil, nil) | }) | github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x0?, 0x0?}) /opt/app-root/src/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.27.4/internal/node.go:524 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() /opt/app-root/src/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.27.4/internal/suite.go:945 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 115 /opt/app-root/src/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.27.4/internal/suite.go:910 Goroutines of Interest goroutine 684 [select, 2 minutes] github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0005f82a0, {0x497bdc8, 0xc001455030}, 0x1, {0xc00193ee40, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:558 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0005f82a0, {0x497bdc8, 0xc001455030}, {0xc00193ee40, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:145 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForSucceededPRCount(0xc001a5cc00, {0x40730e9, 0x23}, {0x3ff36a7, 0x5}, {0x402ad13, 0x13}, 0x1, 0x1a3185c5000, 0x6fc23ac00) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:104 | namespace, succeededCount, expectedCount, displayType) | return succeededCount > }, timeout, poll).Should(Equal(expectedCount), | "expected %d successful %s PipelineRuns in namespace %s", | expectedCount, displayType, namespace) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains.func1({{0x40730e9, 0x23}, {0x409c952, 0x2b}, {0x404c89a, 0x1b}, {0x406de14, 0x22}, {0xc0014c6db0, 0x12}, ...}, ...) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:165 | By(fmt.Sprintf("Waiting for build PipelineRun for %s in %s (base: %d)", | component.Name, tenant.Namespace, base.build)) > waitForSucceededPRCount(fw, tenant.Namespace, "build", component.Name, | base.build+1, PipelineTimeout, PipelinePoll) | > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains in goroutine 683 /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:156 | for _, comp := range Components { | wg.Add(1) > go func(tenant Tenant, component ComponentDef) { | defer GinkgoRecover() | defer wg.Done() goroutine 685 [select, 2 minutes] github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0005ef420, {0x497bdc8, 0xc0015be420}, 0x1, {0xc0019a0dc0, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:558 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0005ef420, {0x497bdc8, 0xc0015be420}, {0xc0019a0dc0, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:145 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForSucceededPRCount(0xc001a5cc00, {0x40730e9, 0x23}, {0x3ff36a7, 0x5}, {0x403b422, 0x17}, 0x1, 0x1a3185c5000, 0x6fc23ac00) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:104 | namespace, succeededCount, expectedCount, displayType) | return succeededCount > }, timeout, poll).Should(Equal(expectedCount), | "expected %d successful %s PipelineRuns in namespace %s", | expectedCount, displayType, namespace) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains.func1({{0x40730e9, 0x23}, {0x409c952, 0x2b}, {0x404c89a, 0x1b}, {0x406de14, 0x22}, {0xc0014c6db0, 0x12}, ...}, ...) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:165 | By(fmt.Sprintf("Waiting for build PipelineRun for %s in %s (base: %d)", | component.Name, tenant.Namespace, base.build)) > waitForSucceededPRCount(fw, tenant.Namespace, "build", component.Name, | base.build+1, PipelineTimeout, PipelinePoll) | > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains in goroutine 683 /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:156 | for _, comp := range Components { | wg.Add(1) > go func(tenant Tenant, component ComponentDef) { | defer GinkgoRecover() | defer wg.Done() goroutine 686 [select, 2 minutes] github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0006da9a0, {0x497bdc8, 0xc0015aac40}, 0x1, {0xc00047c8c0, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:558 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0006da9a0, {0x497bdc8, 0xc0015aac40}, {0xc00047c8c0, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:145 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForSucceededPRCount(0xc001a5cc00, {0x40730e9, 0x23}, {0x3ff1e87, 0x4}, {0x4021f6c, 0x11}, 0x1, 0x1a3185c5000, 0x6fc23ac00) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:104 | namespace, succeededCount, expectedCount, displayType) | return succeededCount > }, timeout, poll).Should(Equal(expectedCount), | "expected %d successful %s PipelineRuns in namespace %s", | expectedCount, displayType, namespace) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains.func1({{0x40730e9, 0x23}, {0x409c952, 0x2b}, {0x404c89a, 0x1b}, {0x406de14, 0x22}, {0xc0014c6db0, 0x12}, ...}, ...) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:170 | By(fmt.Sprintf("Waiting for test PipelineRun for %s in %s (base: %d)", | component.Name, tenant.Namespace, base.test)) > waitForSucceededPRCount(fw, tenant.Namespace, "test", component.Name, | base.test+1, PipelineTimeout, PipelinePoll) | }(t, comp) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains in goroutine 683 /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:156 | for _, comp := range Components { | wg.Add(1) > go func(tenant Tenant, component ComponentDef) { | defer GinkgoRecover() | defer wg.Done() goroutine 687 [select, 2 minutes] github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0005ef1f0, {0x497bdc8, 0xc0015be310}, 0x1, {0xc0019a0d40, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:558 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0005ef1f0, {0x497bdc8, 0xc0015be310}, {0xc0019a0d40, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:145 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForSucceededPRCount(0xc001a5cc00, {0x406de36, 0x22}, {0x3ff36a7, 0x5}, {0x402ad13, 0x13}, 0x1, 0x1a3185c5000, 0x6fc23ac00) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:104 | namespace, succeededCount, expectedCount, displayType) | return succeededCount > }, timeout, poll).Should(Equal(expectedCount), | "expected %d successful %s PipelineRuns in namespace %s", | expectedCount, displayType, namespace) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains.func1({{0x406de36, 0x22}, {0x4097168, 0x2a}, {0x404820c, 0x1a}, {0x4069253, 0x21}, {0xc001385218, 0x12}, ...}, ...) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:165 | By(fmt.Sprintf("Waiting for build PipelineRun for %s in %s (base: %d)", | component.Name, tenant.Namespace, base.build)) > waitForSucceededPRCount(fw, tenant.Namespace, "build", component.Name, | base.build+1, PipelineTimeout, PipelinePoll) | > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains in goroutine 683 /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:156 | for _, comp := range Components { | wg.Add(1) > go func(tenant Tenant, component ComponentDef) { | defer GinkgoRecover() | defer wg.Done() goroutine 688 [select, 2 minutes] github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0005ef6c0, {0x497bdc8, 0xc0015be500}, 0x1, {0xc0019a0e40, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:558 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0005ef6c0, {0x497bdc8, 0xc0015be500}, {0xc0019a0e40, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:145 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForSucceededPRCount(0xc001a5cc00, {0x406de36, 0x22}, {0x3ff36a7, 0x5}, {0x403b422, 0x17}, 0x1, 0x1a3185c5000, 0x6fc23ac00) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:104 | namespace, succeededCount, expectedCount, displayType) | return succeededCount > }, timeout, poll).Should(Equal(expectedCount), | "expected %d successful %s PipelineRuns in namespace %s", | expectedCount, displayType, namespace) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains.func1({{0x406de36, 0x22}, {0x4097168, 0x2a}, {0x404820c, 0x1a}, {0x4069253, 0x21}, {0xc001385218, 0x12}, ...}, ...) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:165 | By(fmt.Sprintf("Waiting for build PipelineRun for %s in %s (base: %d)", | component.Name, tenant.Namespace, base.build)) > waitForSucceededPRCount(fw, tenant.Namespace, "build", component.Name, | base.build+1, PipelineTimeout, PipelinePoll) | > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains in goroutine 683 /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:156 | for _, comp := range Components { | wg.Add(1) > go func(tenant Tenant, component ComponentDef) { | defer GinkgoRecover() | defer wg.Done() goroutine 721 [select, 2 minutes] github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc00049f7a0, {0x497bdc8, 0xc0014554b0}, 0x1, {0xc0018d27c0, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:558 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc00049f7a0, {0x497bdc8, 0xc0014554b0}, {0xc0018d27c0, 0x4, 0x4}) /opt/app-root/src/go/pkg/mod/github.com/onsi/gomega@v1.39.0/internal/async_assertion.go:145 > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForSucceededPRCount(0xc001a5cc00, {0x406de36, 0x22}, {0x3ff1e87, 0x4}, {0x4021f6c, 0x11}, 0x1, 0x1a3185c5000, 0x6fc23ac00) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:104 | namespace, succeededCount, expectedCount, displayType) | return succeededCount > }, timeout, poll).Should(Equal(expectedCount), | "expected %d successful %s PipelineRuns in namespace %s", | expectedCount, displayType, namespace) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains.func1({{0x406de36, 0x22}, {0x4097168, 0x2a}, {0x404820c, 0x1a}, {0x4069253, 0x21}, {0xc001385218, 0x12}, ...}, ...) /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:170 | By(fmt.Sprintf("Waiting for test PipelineRun for %s in %s (base: %d)", | component.Name, tenant.Namespace, base.test)) > waitForSucceededPRCount(fw, tenant.Namespace, "test", component.Name, | base.test+1, PipelineTimeout, PipelinePoll) | }(t, comp) > github.com/konflux-ci/e2e-tests/tests/disaster-recovery.waitForPipelineChains in goroutine 683 /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/tenant_application_lifecycle.go:156 | for _, comp := range Components { | wg.Add(1) > go func(tenant Tenant, component ComponentDef) { | defer GinkgoRecover() | defer wg.Done() ------------------------------ SSSSSSSSSSSSSSSSSSS Summarizing 7 Failures: [FAIL] [enterprise-contract-suite Conforma E2E tests] test creating and signing an image and task verify-enterprise-contract task [It] succeeds when policy is met [ec, pipeline] /tmp/tmp.dle8pz85Yc/tests/enterprise-contract/contract.go:190 [FAIL] [integration-service-suite Status Reporting of Integration tests] with status reporting of Integration tests in CheckRuns when a new Component with specified custom branch is created [It] should lead to build PipelineRun finishing successfully [integration-service, github-status-reporting, custom-branch] /tmp/tmp.dle8pz85Yc/tests/integration-service/status-reporting-to-pullrequest.go:144 [PANICKED!] [upgrade-suite Create users and check their state] [It] Verify AppStudioProvisionedUser [upgrade-verify] /opt/app-root/src/go/pkg/mod/k8s.io/apimachinery@v0.34.2/pkg/util/runtime/runtime.go:114 [FAIL] [build-service-suite Build service E2E tests] test git provider fj PaC component build when the component is removed and recreated (with the same name in the same namespace) [It] should no longer lead to a creation of a PaC PR [build-service, github-webhook, pac-build, pipeline, image-controller, forgejo, build-custom-branch] /tmp/tmp.dle8pz85Yc/tests/build/pac_build.go:732 [FAIL] [build-service-suite Build service E2E tests] test git provider fj component update with renovate when components are created in same namespace [It] should lead to a nudge PR creation for child component [build-service, renovate, multi-component, forgejo] /tmp/tmp.dle8pz85Yc/tests/build/renovate.go:427 [FAIL] [build-service-suite Build service E2E tests] test pac with multiple components using same repository when components are created in same namespace [It] the PipelineRun should eventually finish successfully for component python-component-sjporj [build-service, github, pac-build, multi-component] /tmp/tmp.dle8pz85Yc/tests/build/multi_component.go:161 [TIMEDOUT] [disaster-recovery DR Same-Version Backup/Restore] when creating tenants and running initial pipelines [It] should wait for all build PipelineRuns to succeed [disaster-recovery, Serial] /tmp/tmp.dle8pz85Yc/tests/disaster-recovery/dr_same_version.go:60 Ran 310 of 463 Specs in 5264.464 seconds FAIL! - Suite Timeout Elapsed -- 303 Passed | 7 Failed | 88 Pending | 65 Skipped Ginkgo ran 1 suite in 1h30m3.031827179s Test Suite Failed Error: running "ginkgo --seed=1776792694 --timeout=1h30m0s --grace-period=30s --output-interceptor-mode=none --no-color --json-report=e2e-report.json --junit-report=e2e-report.xml --procs=20 --nodes=20 --p --output-dir=/workspace/artifact-dir ./cmd --" failed with exit code 1 make: *** [Makefile:25: ci/test/e2e] Error 1